So check this out, guys. Meta just dropped their latest machine learning model called Code Llama. And get this, it’s designed specifically for generating software source code. How cool is that?
Now, Code Llama is part of the Llama family of language models. That’s why you’ll see the occasional capitalization “LLaMA.” It’s actually based on the Llama 2 model that was released back in July. But here’s the twist – Code Llama has been fine-tuned and trained to generate and discuss source code instead of regular old text like its predecessor.
Let me tell you though, like any cutting-edge tech, Code Llama does come with some risks.
But here’s the deal, if you’re planning to use Code Llama, make sure you communicate with it in English. As of now, it hasn’t gone through safety testing in other languages. So who knows what it might say if you hit it up in a language it’s not familiar with.
Now, here’s the thing about Code Llama, and Meta is very clear on this. They’re not trying to downplay the risks. They fully acknowledge that any cutting-edge technology like this comes with its fair share of risks. But here’s something interesting – during their red team testing to see if Code Llama would generate malicious code, it actually came up with safer answers compared to ChatGPT (GPT3.5 Turbo). So that’s promising, right?
Meta claims that Code Llama performs better than other open-source code-specific LLMs and even its own parent model, Llama 2. It’s been put through two benchmarks – HumanEval and Mostly Basic Python Programming (MBPP) – and it matches the performance level of OpenAI’s ChatGPT.
So here’s what we got in terms of sizes – Code Llama comes in three sizes: 7B, 13B, and 34B parameters. And let me tell you, each variant has been trained with a whopping 500B tokens of code and code-related data. Just to give you an idea, one token is about four characters in English. That’s pretty massive. And you know what’s even more impressive? The largest version of OpenAI’s Codex, when it was first released, had just 12B parameters.
Now, let’s dive a bit deeper into the different versions of Code Llama. So the two smaller models have been trained to fill in missing source code. This means they can be used for code completion without any further fine tuning. On the other hand, the 34B version is the real deal. It provides the best results overall. But here’s the thing, the smaller models respond faster, which makes them perfect for tasks like code completion where you need that quick response time.
Now, within Code Llama, you’ve got two variants to choose from. There’s Code Llama – Python, and then there’s Code Llama – Instruct. The Python version is fine-tuned with an extra 100B tokens of Python code, making it a solid choice for Python programmers. And the Instruct version has been specifically fine-tuned to follow input and output patterns, making it ideal for generating code.
Now, let’s talk reliability, folks!
I gotta be real with you guys. Sometimes, language models like LLMs can spit out incorrect answers when it comes to programming prompts. But hey, that doesn’t stop many of us developers from using them to get those rote patterns and API parameters, right? Well, the cool thing about Code Llama is that it can handle code sequences with up to 100,000 tokens in them. That means you can feed it with multiple lines of code and get a nice, detailed response. How awesome is that?