A Journey Through Time- The Transformation of AI Development#
I started studying about Machine Learning in 2016, when I took a famous course by Andrew Ng on Machine Learning. After a lot of breaks in between, I was able to complete it and learnt the basics.
My curiosity led me to take courses on Mathematics and Statistics, Deep Learning, then Computer Vision and on earlier versions of language/sequence models like RNN and LSTM when there was no ‘[1]Attention is all you need!’ a research paper by Google which revolutionized the space of AI. To delve even deeper, I even explored classical Image Processing using packages like OpenCV.
Enrolling in courses, building fundamental architectures, training models from the ground up, and experimenting with hyperparameters in PyTorch to achieve high scores in Kaggle competitions has significantly enhanced my comprehension of the foundational principles and the core concepts behind various machine learning paths.
Over time, I observed another rapidly advancing area: GANs. This field serves as the foundation for all modern Generative AI models, including Diffusion models for image generation. The progress in this domain owes much to the pioneering work of Ian Goodfellow, who authored a seminal paper on [2]Generative Adversarial Networks (GANs) while pursuing his Ph.D. under the mentorship of Andrew Ng.
Both the attention mechanism and the Generative AI papers have reshaped our understanding of AI. These papers have triggered a wave of new innovations where context length for Large Language Models (LLMs) is no longer a limitation, and this is achieved without compromising on quality.
With an enormous amount of data available and hardware capabilities within easy reach, along with substantial investments from those who believe in the possibilities of the future, we are witnessing a tsunami of pretrained models. They are being trained on extensive datasets and enhanced with numerous strategies and techniques to excel in various benchmarks.
Of course, we can’t forget the biggest enabler of the surge, ChatGPT, which started the revolution of new players in this field and also enabled development of Open Source models.
Earlier many few used to talk about the language models, because of their limited context lengths and vanishing gradient issues, and Computer Vision used to be the talk of the town , both in terms of research and people participating in Kaggle competitions or even on real use cases applicability. Even the infrastructure boomed around computer vision itself.
With a sudden shift, now we see mostly Generative AI trend around us with a race of pretrained models knocking our doors everyday. With such fast pace development, we needed a fast prototyping ecosystem where different pretrained models can be experimented with different Prompting techniques. And we saw a boom of the platforms where a model can be connected with different prompt styles, vector databases, schemas in just couple of lines of code.
In case of LLM, we see open source projects like Langchain, Llama-Index, various new age vector databases like chroma, pinecone and even no code AI app development companies have started growing, they have not just brought efficiencies but have also enabled faster time to market by providing reusable packages and integrations.
When we hear the word ChatGPT, the immediate thing or only thing which comes in our mind is a Chat bot, because this is the closest we have been familiar with. But, the ecosystem growth around AI has enabled to think and develop beyond an intelligent chatbot. Best example here are Agents.
Core concept of an Agent is , running applications just by chatting with the bots.
Imagine, you want to shop on Amazon directly by conversing through a bot by voice or text, you ask the bot to buy a pair of White Sneakers of XYZ brand, bot asks some questions from you and internally formats the data, uses it on amazon website to search and add the product in cart and do checkout and if you have authorized for bank transaction, your money gets auto debited. Same bot, also tries to find out the best deals from other trusted websites and you might end up buying the same sneaker at a lower price. A fascinating automation of a mundane task
Because of these innovations and frameworks, we are going far beyond the traditional imagination of AI and making the impossibilities a reality. Frameworks like Langchain and model repositories like HuggingFace have eased the development of such applications by providing very easy to use API libraries. In just 10 lines of code , you can create an Open Source model based application that can add Sneaker shoes in your Amazon cart.
There is another stream of development in AI which has started budding, [3] simulated environment of Generative Agents where a group of AI agents can converse with each other in a simulated environment and build a system of their own. For example, you provide a scenario of creating a website for lead generation to a head agent which acts as a Product Manager, it then identifies the task and assign them to sub agents who then work on their respective tasks, they code, collaborate and develop a software which then is tested by Agents only and finally gets delivered to you.
Sounds like a science fiction but it is soon becoming a reality with research papers and POCs being published very frequently on this topic.
We are already seeing huge mushrooming of new age start ups on AI because of the promise of the future being committed to the stakeholders , enormous improvements in hardware capabilities and fastest time to market.
References:
[1]Attention is all you need :https://doi.org/10.48550/arXiv.1706.03762
[2]Generative Adversarial Networks: https://doi.org/10.48550/arXiv.1406.2661
Generative Agents: Interactive Simulacra of Human Behavior: https://doi.org/10.48550/arXiv.2304.03442
Courses and libraries:
Machine Learning Specialization: https://www.coursera.org/specializations/machine-learning-introduction?
Deep Learning Specialization: https://www.coursera.org/specializations/deep-learning
Mathematics for Machine Learning: https://www.coursera.org/specializations/mathematics-for-machine-learning-and-data-science?
Langchain: https://python.langchain.com/
HuggingFace: https://huggingface.co/
Comments
comments powered by Disqus