Publicado em Sept. 11, 2020

The Revolution within the Artificial Intelligence Revolution: the GPT-3

Why you need to know what this new and disruptive milestone in AI means for humanity

Working and researching Artificial Intelligence (AI) has some advantages, one of which is the amount of incredible things that you can see, some of them first hand nowadays. The famous American writer Arthur Clarke, author of the masterpiece immortalized in cinema by Stanley Kubrick: "2001, a Space Odyssey", was very happy to coin the phrase: "Any technology sufficiently advanced is indistinguishable from magic." And that is the feeling I have with many of the innovations that we are seeing happening in the world of AI and its applications.

And even having the opportunity to see and know fantastic things on a regular basis, there are those news that manage to surprise your most optimistic predictions. Using the comparison made by Arthur Clark, knowing what the GPT-3 can do, gives the feeling of pure magic, and the most powerful. So, in the right proportions, for everything I read and heard, this is what the GPT-3 represents to the world: a fundamental technological milestone that has the potential to popularize the most disruptive technology today and make us humans we gain unimaginable productivity. And if you are a person who does not understand anything about programming, maybe you will like even more than this new Artificial Intelligence “architecture” is capable of.

If you haven't heard of the GPT-3, don't worry (yet), it is coming and for now it is only available to some researchers connected to the OpenAI project. OpenAI has gained increasing prominence in the world of technology, and is a non-profit research institution in artificial intelligence, which aims to promote and develop friendly AI, in order to benefit humanity as a whole. Among its founders and investors are multi-billionaire and popular Elon Musk, owner of SpaceX and Tesla, Sam Altman, current CEO of OpenAI and former partner and president of Y Combinator, a successful American accelerator, and Peter Thiel, one of the co-founders from PayPal with Musk himself and founder of Palantir Technologies.

The GPT is an OpenAI project that seeks to improve the understanding of (human) language through the use of NLP (Natural Language Processing, one of the technologies of Artificial Intelligence) and Machine Learning using Deep Neural Networks, also known as Deep Learning. The acronym GPT stands for “Generative Pre-Training” in English, which translates freely to Portuguese as “Generative Pre-Training”.

The idea in general is to guarantee a wide training of this model of Deep Learning with specific bodies of knowledge that allow to understand patterns of requests / requests of human language, and to fulfill these requests / requests through the identification of patterns obtained in this pre-training. It is a unidirectional model where you place an order in natural language (currently in English only), and the model delivers your answer. Pre-training ensures that the model has been previously taught in an unsupervised manner, and thus uses a massive amount of data, information, and explicit knowledge on specific topics for its “learning” widely available on the internet.

In this sense, what has made GPT-3 so disruptive in relation to other NLP architectures and their previous versions, among other things, is the number of parameters. Roughly speaking, a parameter is a calculation in a neural network that applies a greater or lesser weight to some aspect of the data, to give that aspect more or less prominence in the general calculation of the data. We can compare in a very simplified way that the parameter is equivalent to the synapse of a neural network of a biological brain for this model, and this pre-training, the equivalent to human experience, which generates specific weights and connections in this network.

To give you an idea, while the previous version of architecture, the GPT-2 had in its largest version, 1.5 billion parameters, the GPT-3 uses around 175 billion (> 100 times more). The size of the GPT-3 neural network is really impressive. It is 10 times bigger than Microsoft's architecture, Turing NLG, the largest network before GPT-3 that had around 17 billion parameters.

The architecture concept of the GPT-3 is more difficult to understand than it is capable of doing in practice. So, some examples of what is already possible to do with the GPT-3:

· Build a program or application from scratch through your feature requests

· Write a short poem in the style of William Shakespeare with the theme of the Emperor and the “Jester”

· Design a social networking posting app and make it practically functional

· Prepare a report with the requested information in the appropriate form

· Write an email to your customers based on little information such as "confirm the service proposal with a 10% discount", "thank Eduardo" and "make an appointment for tomorrow to finalize the details", the The system completes all the gaps, and places a formal pattern of responses for email clients of this nature.

· Prepare a slide show on a proposed topic

· Write an article about chemistry and the importance of alkalinity in food.

That is, the GPT-3 is a prototype of Genie Lamp for the AI ​​world. You place orders, and he uses the wide range of knowledge about programming, poetry, design, management reports, emails, slide shows and chemistry, among other things, available on the internet. All this cataloged knowledge is used to pre-train this neural network, which has hundreds of billions of parameters, this pre-trained network is then used to build the responses and deliver these requests in minutes.

With great advantages, the GPT-3 has a good understanding of human language, and with few adjustments to requests made it can bring very effective results. It impresses the very low number of examples that it is necessary to provide so that the GPT-3 understands your request quite properly. The GPT-3 hit rate for orders placed without any “example” / order adjustment is quite reasonable, and with a few adjustments your accuracy level grows exponentially.

The GPT-3 is able to build a balance sheet, in real time with a few instructions and information in natural language as in the image below:

Or build a website using this GPT-3 and Figma plug-in:

At some point in time, Microsoft, with Windows 3.0, revolutionized operating systems, definitively impacted the popularity of personal computers, and the future of computing itself. Now a new architecture, also marked by the number "3", has an incredible potential to forever revolutionize Artificial Intelligence, making it easy to access and use for any mortal, including those who have no technical knowledge of programming. And with that, unlock the creative potential of each one of us to generate even more incredible innovations. With the GPT-3, a new era of technological magic is about to begin.

There are a lot of really cool videos out there. Some of the coolest and most different examples have been organized and can be seen on this website: https://gpt3examples.com/#examples. The content is in English.

262 reads 63 Likes

About the author

Alexandre Del Rey
Sempre aprendendo

Alexandre Del Rey

CEO e Founder I2AI

É sócio fundador da I2AI – Associação Internacional de Inteligência Artificial. Também é sócio-fundador da Engrama, sócios das Startups D2i e Egronn e investidor nas startups Agrointeli e CleanCloud. Tem mais de 20 anos de experiência em multinacionais como Siemens, Eaton e Voith, com vivência em países e culturas tão diversas como Estados Unidos, Alemanha e China.
Palestrante internacional, professor, pesquisador, autor, empreendedor serial, e amante de tecnologia. É apaixonado pelo os temas de Estratégia, Inteligência Competitiva e Inovação.
É Doutor em Gestão da Inovação e Mestre em Redes Bayesianas (abordagem de IA) pela FEA-USP. É pós-graduado em Administração pela FGV e graduado em Engenharia Mecânica pela Unicamp.

Read too

262 reads 63 Likes
AI Verticals
Próximo Evento
March 8, 2021

AI Verticals

We already know that Artificial Intelligence impacts all sectors of the economy. Looking by sector allow us to find opportunities and applications in a direct and effective way. While looking