Creating your own models

BIld av Pexels från Pixabay

Last week I wrote about our seminar and Co-pilot. I’m sure that has stimulated a lot of thoughts on these language models. Many think that this is a difficult task to create, train and use them.

Nothing further from the truth. If you are interested in training such a model from scratch, I recommend the following book (in particular Chapter 4 if you are anxious to get started).

Transformers for Natural Language Processing | Packt (

The book explains how these models work for natural language processing, but making it work for source code is trivial. Use your code instead of the provided text and there you go. You need a GPU or use some cloud service, otherwise you will wait forever.

But if you have it, you can get really cool results within a day or two.

Good luck!

Author: Miroslaw Staron

I’m professor in Software Engineering at IT faculty. I usually blog about interesting articles (for me) and my own reflections on the development of Software Engineering, AI, computer science and automotive software.