JSMP 5: Gerard Sans on OpenAI – The Start of a New Era in AI (Transcript)

JSMP-05-transcript.txt

foreign Poland conferences are coming soon
this year promises to be exceptional we will see the angular team on the stage
we will see quick create solid.js Creator experts from Google Microsoft
Amazon Cisco all zero and many many more join an amazing group of developers like
you today let's come together to celebrate angular and JavaScript go to
ngpaulin.pl and sign up now [Music]
what's up everyone this is founder of AMG Poland JS Poland England mastered to
death and workshopfest.dev welcome back to the JavaScript Master podcast today we've
got a special guest from London UK M.C speaker trainer Community leader ladies
and gentlemen Gerard sounds [Music]
[Applause] hi guard how are you
hello hello Eric I'm very good okay so for those who don't know you yet please
tell us about yourself well I'm Gerard Sands I'm a developer evangelist for web
and cloud and I've been doing some research on artificial intelligence and
web 3 for the last year and I'm here to share some of my learnings
Will AI replace our jobs
opening question will AI replace our jobs
well this is this is uh for some people uh very uh scary uh time we have seen a
lot of improvements in the last few years and some of them affect how people
approach their jobs and probably the most significant changes has happened on
text to image generation and this is where we use a prompt and an AI is able
to generate images from scratch and we have seen
[Music] crazy improvements there these AIS can generate illustrations can
generate photorealistic images and it just gets better and better and a lot of
people working as a designers or maybe as a digital
artist they can see that their jobs are on the line and there's no clear
separation of what we as people will
accept from a digital artist or from maybe an AI artwork which of these are
Writers artists and developers
more likely to be affected writers artists or developers well that's
something that it's been very surprising for me I um I started looking at some of
the text that AIS were able to uh to create and this is very similar to what
I was explaining from images but with text and the idea behind this
is that these AI models have been trained using a very large amount of
data and public books from the last few centuries so this is
anything from all the possible styles that you can
imagine commercial books self-help cooking recipes but also philosophy math
and all of the science and if you think
about big authors like Shakespeare or other
um popular authors in history these are also in in these models so actually you
can create text that resembles any of the most famous writers in the world and
of course including marketing and any other
areas so that's that's one one thing that if I was a writer I would use these
tools to maybe avoid what it's called the writer's
block so you can get inspiration of
of a history that you are kind of stuck and you can get some text to continue
a paragraph that you have written but people is using this to generate
anything from a blog post to cooking recipes or anything in between so it's
it's quite it's quite exciting but also it's quite scary for for different jobs
like technical writer or even a novelist
these these are all in the line and most people know about this technology but
they don't think that maybe as developers they will be affected by this
but it was only last year that open AI
it really is a codex which is the model behind GitHub copilot and it's able not
only to write code from scratch but also do refactors which which is basically
anything that you need as a developer you can write a code and then iterate
over it as many times as you need and this is what these tools are doing today
so it's uh it's quite a new era of AIS I would say
What is OpenAI
this is crazy so um yeah completely insane so what is open AI openai is a
company that was founded by Elon Musk son Oldman and Greg Brockman and
what they say about their mission is that they want to build a safe
artificial general intelligence that benefits all humanity and they created
open AI in back in 2015. as a non-profit company
and it took them few years until 2018 to create the first GPT model which is a
generative pre-trained Transformer but then they went ahead and published
and a public API which was in 2020 which is
when everyone started trying these technology and they discovered many
applications the initial ones were around writing but
they also discovered that gpt3 was able to go JavaScript
and that was when uh when they started working in in codex
which was released on 2021 so this is a very recent technology it's just been
around for a couple of years and the problem is just fantastic what products
Products from OpenAI
are being offered today there are three different products and
the one that I was talking at the beginning is gpt3a and this is the model that is
able to generate tax and the way it will work is that we provide a prompt
and gpt3 will continue that prompt in a way that makes
sense with the with the actual text that he founds
and you can use that for example to create a story
but it has also been used to
follow a conversation and this is some of the things that were
not plan this model was created to generate text
and predict the next word following a prom but then they realized that this
model could identify patterns like a series of
questions and answers and then that will start giving you answers back in a way
of uh of a conversation and they found that gpt3 was able to do
a lot of different things you can for example ask gtt3 for cooking
recipes you can ask for how to cook a certain meal and gpt3 will give you the
ingredients and how to uh how to cook it and that's not really
a text generation anymore in the sense of writing a story or writing a book
but it's more like identifying a pattern identifying a question and providing an
answer to that question which was completely unexpected uh for the
scientists that were building this uh these models but then they started doing more tests
and they identified a lot of different usages for for gpt3 and that was kind of
the beginning one of the things that they realized using gpt3 is that it could actually
write HTML and JavaScript and that brings the second product from
open AI which is codex when they realize that gpt3 was able to
write JavaScript they um they train gpt3
only with a source code so that was when openai contact
Microsoft and Microsoft gave access to GitHub and the parts I think 59 million
repositories and it learned a list of 12
different languages including SQL and Bash commands
What is GPT3
well said what is gpt3 trying to solve well gpt3
was um natural language processing model that was intended to
generate tax and the idea behind was
taking the prom and trying to guess what will be the next work that follow that
problem the thing is you can repeat that sequence so you can generate as many
words as as you need and then that will be the main operation for gpt3 so you
can provide a text as an input and then it can provide any number of words
usually what you would do is give it a stop sequence so for example a new line
will be the stop sequence so it will stop after the first paragraph but you
can you can create complex patterns as a chat so you can make a question and
answer text and gptc will follow that or you
can ask for instructions and it will provide the instructions to achieve a goal
like cooking a recipe cooking recipe sounds
GPT3s Languages
so weird for me created by Ai No it's uh it's it's quite
it's quite um new the idea behind is that these
models are very large models and
gpt3 is actually 175 billion parameters
uh large is is the size of the whole model but that means that it has more
parameters than the size of the atoms of the universe so this is a very this is a
very big model that we cannot imagine and it has a lot of things inside and of
course one of the things it has is a cooking recipes but this is not this is
not the most weird thing that you can find inside this model because it
literally codes in JavaScript but not only JavaScript I was
mentioning some of the languages it learned but it has learned
typescript JavaScript python Ruby go C sharp Swift rust
PHP all of these languages after just a
little bit less of half a terabyte of information
but this is also not the the most weird thing probably the most weird thing is
that it it can speak more than a hundred languages when it was only trained in English
which is also very surprising some languages gpt3 learned with less
than one percent of uh data which I don't think nobody knows how
that happened it was supposed to learn English but because there were like some you know uh
in the data in the training data it took information from different sources and
one was a common crawl so it took the top thousand websites
in Alexa and it took all the content and that's how it learned JavaScript because
a lot of the raw content of a common crawl is HTML and JavaScript but then it
also took hundreds of years of public books and
funny enough it also took the information from Reddit
so there's a lot of information from users and the links that these users provided
so there's anything any any information you can imagine is in there
and then probably one other resource that was in the training data was
Wikipedia so anything that is in Wikipedia is also knowledge of gpt3
so it's a massive amount of data and yeah it can do funny things like
translate between languages so you can write something in French and it can give you the translation in Polish
why not it's it's completely it's completely insane and if you look at the
complexity of a language like English then you can understand how it's able to learn JavaScript because JavaScript is
much easier than it's yeah yeah exactly exactly but if it has
learned uh languages like Japanese yeah if
how the question is more like how how we'd learn Japanese with almost no
information yeah um and we're gonna see more I think that
this is just at the beginning and we haven't seen uh all the all the power I mean we have
seen a little bit we we have seen like the tip and uh because people hasn't got time to
experiment we are talking of uh of something that it's massive
um in artificial intelligence that's uh called latent space so this this phase
of 175 billion parameters is so big that you will not have time in your entire
life to explore it um where we are talking about uh gpt3
giving you cooking recipes this is only a small uh grain of salt in the imagine
if it was a beach or maybe all of the beaches you know the whole world the
sand it will be just taking like a little a little uh grain of of sand I
mean this is nothing compared to the size of of the whole gpt3 so the the
idea that I want to share with uh the people is that we haven't seen anything
I mean we have seen some things that are surprising because nobody was expecting
a model to understand um JavaScript but now it's been more than
half a year being used in production with GitHub copilot and the service is
still running so I guess it's working so it's giving code to professional
programmers around whatever they are building I mean that's that's very impressive
CodeX
you're listening JavaScript Master podcast listen code repeat everything
you need to know to become an JavaScript super developer
while these codex's main purpose and for for codex is
generating uh code so we have talked about gpt3 being able to write as
Shakespeare which is a quite quite fun but it can also write in any in any
style that you can imagine so if you if you can find enough information
which gpt3 can have been trained on so it must be a popular writer it cannot be
someone that has uh not many books or materials but for codex is anything that
it's on GitHub and part of the training was all of these 59 million repositories
so you can imagine that anything in GitHub it will be there so codecs can
help you use all of these libraries the only thing that you need to take into
account is that codex was trained with a snapshot
so it's not being trained daily because it's very expensive
process but it was trained last year so there's a snapshot of last year whatever
was in GitHub last year so that that means that it's angular version 13. it's not
angular version 14 because that happened after the release was after so anything that
it's after the snapshot codex is not aware of
what is Dali and how people are using it
well Dali is probably the most popular of the of the three Dali is everywhere
is social media is on Tick Tock is on Twitter every day there's people that
goes into Dali and generates an image so you can you can see a new image every
day if you follow these users Dali is following this Trends so we have seen
how these models generate text how this model generate code and this is exactly
the same for images the only thing that they did here is
they took all the knowledge around languages so everything that gpt3 knows Dali knows
that means that it's a writer it's also
a developer that knows all these 59 million GitHub repositories but now it
has this information mixed with images and they have trained
gpt3 for Dali with around 400 million
images which is completely insane again and the main
the main feature that it can it can
it can generate any sort of images following the same as gpt3 and codecs
you need to provide a prompt and then from this problem it will generate the
image following the instructions and this is this is very surprising because
you can not only generate image from from scratch but you can also edit
images and give instructions so for example one
of the experiments that I did recently was to create a photorealistic picture of a woman
which can create no problem for tally so
it creates the picture following all your instructions you can provide
information from the composition how the model is positioned in depiction what is
the lining what is the camera that it's being used and
I I got something no I got the picture I was after and then I changed the image
and ask tally to add I black leather
jacket to the to the model and it did so you can you can ask things
as if it was some kind of a system that is helping
you to create these uh this setup um so you can ask things like at
sunglasses to the model or maybe add a scarf
but not only it will add these elements to the picture but it will take into
consideration the volume it will take into consideration the lining it will
just place that item in the model and give you the resulting image which
is completely insane and it cannot it cannot only do
photography because photography is probably the most difficult but you can do pixel art it can do 3D models it can
do games it can do illustrations of course photography is completely crazy
but you can do like drawings like hand drawings uh
it's it's it's really something it's really something else people is also
creating um games um because games usually when you need
to create a 3D model object it's quite um it's quite a lot of work
and sometimes you need to create a texture imagine you are creating a
castle and there's some pattern in the in the walls which is uh from Brock I
mean imagine there's some there's something else maybe it's wood it's a wood wall and it's difficult for an artist to
create this kind of textures but you can give it to Dale and it will create the
texture for you you can then use it in a 3D model and in a few minutes you have
got everything and you are ready to go there's people building games like super fast these
days that sounds sounds really amazing yeah there's also some fancy uh use
cases one of them is for example now that I can see you uh in the call I can
see that you have like a a sofa in the background so you could take a picture
people is taking picture of a room and you can ask dally to add Furniture
so imagine there's something missing maybe you want to add some artwork in
the box like some pictures so you can you can give this task to Dali and it
will just decorate your room in different styles if you want
people is also using Dali to recreate historic
scenes like imagine you want a picture from
Normandy you know normally when in the second world war yeah so you can ask
Dali to give you a selfie from a soldier in Normandy it's it's
anything that you can imagine dollies is able to use all the
information that you give it it's more like a die a direct a creative director
and it just puts all the information uh
it makes and it makes it work of course the Technologies is very new we are
talking about a technology that last year didn't exist so in just a year it
has done a lot of improvements so we will see many many other things happening in all of these different
products but Dali is probably the most because it's visual it's the most
surprising can you use open night today how would
OpenAI Today
that work well there's a there's an API that you
can use some of the apis are open to anyone tpt3 is completely open everyone just
needs to go create an account and start using it and the only thing is that it's
not for free so you will have to buy some credits and then
activate that access with the API key
like you Google with other services and then if you want to have access to
codecs you need to apply for a for a better beta access
and also for that dally takes a little bit longer so if you apply today it will
take a little bit longer but you can use all of these Technologies without
without any problem what are the benefits of using those tools are they
Benefits and Drawbacks
any drawbacks I think the benefits today is that they will boast your efficiency
so some of this some of the tasks that you can do with gpt3 like for example
imagine you want to create a technical blog post that will give you like a big
post so if you needed two days to edit
your your text and go through different revisions with gpt3 you can probably do
that in in few hours or maybe in a couple of hours with the other products
it may depend what what is that you want to do sometimes the quality is not there
yet but you can you can use parts of the of the results of the outputs to speed
up your project so you you can get uh more efficient you can do things faster
I I mentioned for gaming how they can create these textures that before you
could have took days for digital artists to create so it just depends on what
exactly is that you want to use it for but yeah it's going to give you that productivity
um as drawbacks I would say that depending of how you use it there's no
easy way to know if the information that maybe egpt3 is giving you it may not be
factual so if you are writing a blog post and some of the information is not
factual it can bite you back because somebody will find out and then you will
be embarrassed that you have used a source that is not it's not a search
so it's not something that you can trust so probably depending how much you rely
on AI you need to check the facts you cannot just take the output from
gpt3 and use it in a publication just uh release it you will have to to check the
facts or you may be embarrassed for other things it just depends what you
try to do sometimes when you try to do something very complex there's some
issues with the technology is not perfect so you can get some artifacts for example in in Dali when it
creates an image faces like human faces and
human models in the pictures they don't look totally they look odd at times sometimes
you need to edit the pictures so they are not all the time like 100 perfect
but I think this these errors or these
drawbacks are gonna be fixed and then it's going to be much much faster to use
these tools the million dollar question what the future of AI may look like in
The Future of AI
five years I I just did I just did the talk around
this topic and I describe um the AI Revolution so this is um from
the work of different people and the first stage it's called cognitive
Revolution and what is the cognitive Revolution this is where we use AI
assistance to get our job done faster and depending of what you do that will
work best or or not I mean even for GitHub copilot it's not that the tool is
creating everything for you that is just giving you some kind of um a
productivity uh boost then The Next Step which is a little bit more
crazy and it's probably not going to be uh that soon so this one using AIS
systems is going to be very soon some people is already using this today but
the next step is the robotics Revolution this means that we will see the first
robots using Ai and being of tournaments
and maybe we have seen some of these autonomous robots I think I saw the
first one in in Silicon Valley some years ago they go around a building and
they can give you information around what you can visit
but of course this is not something that you can find everywhere but in the
future we will see more examples of autonomous robots and this is the second
stage the third stage is when AIS get
smarter than humans and this is called the singularity
and when we reach the singularity this is uh similar to the character in Star
Trek Data I don't know if you if you remember data from Star Trek this is a
Android that helps Star Trek characters when they when they
struggle and they go into Adventures when they have questions about what they
are seeing like maybe there's some danger and they need to decide make a decision if they they go into a planet
or not then they will ask data and data will be like some kind of
uh super intelligent being that will give some advice in that situation and
this is similar also uh to the intelligence of a spook uh character
so these are some these are some some of the things that may happen in this AI
Revolution and and people is is not um
certain when all of these different stages will happen but uh there's a
quote from Elon Musk that was from not
long ago and he said that we were going to be replaced by AIS in
the next five years so that's that's what Elon Musk uh has to say around this
topic do you think AI can help us build something we can't do ourselves for now
AI for Rocket Engines
like a super engine for a rocket oh yeah definitely definitely yes
um what what is happening is that with these models we have created some kind
of uh some kind of a super intelligence not not in the sense of because there's also
the debate if they are sentient or if they are autonomous or if they are like
uh some kind of a god entity I mean I don't think they are a god entity or
anything like that but the the main thing is that they have all the
information accessible at the same time so one of these uh
technologies that is being used behind the scenes is neural networks and a
neural network it's not doing calculations all the calculations are
already done and that's how these neural networks are so fast that you can use them for
example in self-driving cars and other systems they are in the range of
milliseconds of the response time so they are they don't need to calculate they are not it's not working like a
search imagine that some people think that these AIS they work like a search engine so they have a query and there's
some computation and then it's coming back with an answer but it's not actually working like that the neural
networks is more like a human it's it's not even thinking it's giving you the
answer right away when you use the systems gpt3 and codex it's not taking seconds not
even milliseconds to answer back and that means that all of the information is accessible to them like
right now it's they don't have to think because that's part of the training
process which is very expensive and it takes weeks but one the training is
finished the model is already is already done
and that means that it has all the skills at the same time so when you access gpt3 you are talking to
something that is it has Shakespeare inside but it also has everything else
it also it's also a developer it's also a chemist it's also a mathematician it's
also it has everything so when you ask him for example uh how
we will build a rocket engine it will give you the answer right away it's not
something that he needs to think like okay I need to think now he will give you the answer right now
the moment you ask it's not it's not gonna tell you oh let me let me uh think
about it and next week I'll give you an answer no no it gives you the answer right now and this is for anything when
it looks at code when it looks at uh at the image or text the model is giving
you the answer in the same moment so that means that it has access to to all
of the knowledge all of the knowledge that it has been trained of course if the if the
information is not in the training data it's not like it's it's a god it's not a God but it's very capable
it's like very very capable but it's not it's not a god some kind of
a new um but it has it has all of the information
the same as it could be Google but it can work with the information it's not
only um it's not only like a library where you have the information and then it retrieves the information for you it has
all the mix of all the all the information and it has the results
um at that same moment so it doesn't it doesn't do calculations intermediate calculations so it's quite impressive
it's a new technology and we are still discovering uh how we can use it so what
the conclusion oh the conclusion is that I don't see nobody knows what is
happening with uh with these models but it's exciting because people find new uh
use cases and they find new things that you can use these models and there has
also been news around open sourcing some of these Technologies so you can use
them in your projects so that's also very exciting after all these discussions can you give some advice to
Advice for New and Old Timers
people who are starting their career in it world today
and for those who are Old-Timers I think I think a good advice is to to keep your
mind open because everything is changing everything is changing people think that
things don't change but in in the area of AI this is uh probably the the better
example AI was is not a technology it's not a area that has been changing a lot
and it's kind of an old area but the
discovery is done in the last few years they have exploded and this is new
it's not there it's more like JavaScript you know they're all JavaScript and the new JavaScript that happened a few years
back and people were surprised like oh can you do this with JavaScript I mean before we didn't use JavaScript in this
way so that's a little bit of a new generation of AI and this is this this will change
everything and a lot of the jobs that we do today they won't exist in the future
so if you are not retiring uh I mean if you are an old-timer you maybe still
have five ten years or 15 years more of career I think you should you should
look into what's happening in in the blockchain what's happening in web3 keep
your mind open because I mean we can become obsolete like very fast for the
Alzheimer's for the new uh Generations I think that they also need to be careful
where they put their energy because if they put their energy learning a
specialization and then they realize that anyone could be able to use any
technology because they will use AI assistance they will be wasting time why why you want to
memorize things or understand things when an AI can help you and you can use
their eye or anyone so then it's not it's not clear that you want to go
through a specialization of 15 years and then find out that someone that has no
specialization is using a tool that is making that instead of 15 years is just
five years so then you have lost 10 Years Learning something that it wasn't necessary
um so that's that's something to to be careful let's change the subject a bit
Last visit to Poland
how do you remember your last visit to Poland OH I am
I I remember as a crazy as a crazy time I mean it's always it's always fun
and it's more like a wheel wheel so it all starts for me because I was doing a
workshop and also participating both in NG Poland in Gs Poland that was uh
that was a crazy A crazy three days and
yeah I I really have a great time meeting all the all friends and also
people that comes and asked me uh questions probably the most surprising
from last year was the award that I received for the community contributions
to JavaScript so that was very special and then also all the fun we have in the
in the back States and we we also make uh jokes and I remembered and that Derek
put me in the spot few times and I was like what's going on I was going crazy
because I didn't know what was happening but this is uh also one of the things
that I that I like from from the conference my last question for today
JSMP 2022
what we can expect at your emceeing at NG Poland and JS Poland
2022. oh it's gonna be full of surprises and I think there will be some
artificial intelligence uh coming but I don't think I can I can share it with
you at this time but the experiments uh I've shared with tarek
and and he was pretty was pretty surprised so we'll see if I can I can
make it happen during the conference otherwise as always uh I will make the
attendees do something it could be jumping it could be uh doing a squat I
don't know we'll do a Mexican wave I mean who knows what will happen on the day
Gerard thank you so much for joining today's podcast and sharing this
exciting knowledge with us I look forward to our meeting in Walsall soon
yeah thanks a lot for inviting me and I'm really looking forward to another
edition of the conference thank you so much bye bye bye bye
everyone finally please subscribe to our podcast
leave a like and a comment to help us continue to grow
foreign [Music]