Jun 26, 2022

--

Very nice reflection,

It's always good to experiment with new approaches and ideas.

However, When Bert was created by Google, they make the choice to only encoder layers, as on the other side GPT is based only on decoder layers. Basically, this choice depends on learning objectives that do not need a decoder. So, did you really need a decoder in your case? I don't think.

--

--

Veritas AI
Veritas AI

Written by Veritas AI

Welcome to Veritas AI, where we pioneer ethical and explainable AI. Our lab focuses on developing AI that upholds transparency and fairness. Follow us on Medium

Responses (1)