Follow and Support me on these Media Channels
As an Open-Source advocate, the announcement of the release of Meta's Llama 2 Large Language Model(LLM) caught my attention. The 1991 decision by Linus Torvalds to Open Source his Linux operating system kernel marked a significant milestone in the history of business computing. Making the source code of the Linux Kernel available to the worldwide community of software developers laid the groundwork for developing the platform that powers most business enterprise back-end platforms.
Meta's decision to release Llama 2 as an Open-Source product will significantly benefit business enterprises of all sizes, worldwide. This decision lays the groundwork for developing a freely available LLM platform that will level the playing field for entrepreneurs, developers and business enterprises worldwide.
Llama 2 is the second version of the open-source language model from Meta. It is based on a transformer architecture and has now also been released for commercial use. This presents many new possibilities, as a variety of applications can be built on the Llama architecture.
Llama stands for Large Language Model Meta AI. It is an auto-regressive large language model that uses an optimized transformer architecture. It is the second Foundational Model from Meta AI, which was released in 2023.
The first version of Llama, released in late February 2023, was already open source. Llama 2 can also be used for commercial applications, opening the door for innovation and expansion of this LLM for a broad range of business applications.
There are several quality Llama 2 introductory videos from reputable sources that provide more information and business use cases for the Llama 2 platform. I'll be updating the subscriber article with additional resources, including how to install Llama 2 on your own hardware platform for private business and professional practice model use cases.
Llama 2 comes in different sizes with 7, 13, 34, or 70 billion parameters. It was trained on 2 trillion tokens with over a million human annotations. Context length has doubled from version 1 to 2, from about 2000 to about 4000 tokens. Besides the size variants, there’s also a fine-tuned variant of the model for chat applications called Llama 2-Chat.
Compared to other open-source models, Meta's new model performs excellently. This moves it to the top stop in the Hugging Face Open LLM Leaderboard. However, compared to proprietary models, like OpenAI’s GPT models, there’s still room for improvement. In some benchmarks, Llama 2 outperforms GPT-3.5, the model based on ChatGPT. For example, in the Hella Swag benchmark, LLaMA-70B-chat performs better than GPT-3.5.
Llama 2 was initially trained in the pre-training phase using publicly available online sources, with a much larger dataset than Llama 1. After pre-training, the first version of Llama 2 chat was created through supervised fine-tuning. Human experts helped with the training at this stage.
To further improve the model’s performance and create more natural responses, Reinforcement Learning from Human Feedback (RLHF) was implemented in the next step. This involves iteratively refining the model with reinforcement learning and human feedback.
At the same time as the Llama 2 release, Microsoft announced itis expanding its partnership with Meta. Llama 2 is available in the Azure AI model catalog, enabling developers using Microsoft Azure to build with it and leverage their cloud-native tools for content filtering and safety features.
Llama 2 is optimized to run locally on Windows, giving developers a seamless workflow as they bring generative AI experiences to customers across different platforms. Llama 2 is available through Amazon Web Services (AWS), Hugging Face, and other providers.
Meta has put exploratory research, open source, and collaboration with academic and industry partners at the heart of their AI efforts.
They have also created new initiatives to harness the insight and creativity of individuals, researchers, and developers around the world to get feedback on how the models are performing and how they might be improved. They have also put a policy in place that prohibits certain use cases to help ensure that these models are being used fairly and responsibly.
If you're a JimIntriglia.com Subscriber, check out the in-depth companion article for free web platforms where you can take the Llama 2 LLM for a spin.
Llama 2: New Open-Source Language Model from Meta Released. (Alexander Goschin, TechMio Blog, 7/21/2023).
Meta and Microsoft Introduce the Next Generation of Llama. (Meta, 7/18/2023)
