Integrating Large Language Models into Applications

Large Language Models have become quite popular in the last 12 months with OpenAI's closed source ChatGPT and Meta's open source Llama releases. Outside the chat interface you might be used to, these models can be called programatically and doing so allows us to easily add Natural Language Processing capabilities to our applications.

Considered prompting of these models using a templated prompt with dynamic inputs can allow us to generate new datasets, edit existing ones, or even generate code. A number of open source models are available but the approach is similar with commercial models too: Load the model, expose it via an API, and then make inference requests to it from your application.

In the following post I'll show how to download and load an LLM ready for inference, construct a prompt and query the model for a response, and parse the response back into our line of exeuction: Data Processing with Large Language Models.