This cheap LLM API was developed by Meta and released the Meta Llama 3 family of large language models (LLMs), a collection of pre-trained and instruction-tuned generative text models in 8 and 70B sizes. The Llama 3 instruction-tuned models are optimized for dialogue use cases and outperform many of the available open-source chat models on common industry benchmarks.
**Input:** Models input text only.
**Output:** Models generate text and code only.
**Model Architecture:** Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.
Params
Context length
Token count
Knowledge cutoff
70B
8K
15T+
December, 2023
**Intended Use Cases** Llama 3 is intended for commercial and research use in English. Instruction-tuned models are intended for assistant-like chat, whereas pre-trained models can be adapted for a variety of natural language generation tasks. This is an easy-to-use LLM API and cheap LLM with cost of $1.20 per million tokens.
**Carbon Footprint Pretraining utilized a cumulative** 7.7M GPU hours of computation on hardware of type H100-80GB (TDP of 700W). Estimated total emissions were 2290 tCO2eq, 100% of which were offset by Meta’s sustainability program.
Time (GPU hours)
Power Consumption (W)
Carbon Emitted(tCO2eq)
6.4M
700
1900
### Training Data
**Overview** Llama 3 was pre-trained on over 15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 10M human-annotated examples. Neither the pretraining nor the fine-tuning datasets include Meta user data.
**Data Freshness** The pretraining data has a cutoff of March 2023 for the 8B and December 2023 for the 70B models respectively.
### Request and Response
#### Request
{% code overflow="wrap" %}
```bash
curl -X 'POST' \
'https://prod.api.market/api/v1/bridgeml/meta-llama3-70b/bridgeml/meta-llama3-70b' \
-H 'accept: application/json' \
-H 'x-api-market-key: API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"messages": [
{
"role": "user",
"content": "hello"
},
{
"role": "assistant",
"content": "As a senior software engineer create a AWS python lambda that calls an API with the given curl request and returns a response, take the input from the function parameters called events, example curl request:\r\ncurl -X GET \\\r\n '\''https://api.magicapi.dev/api/v1/magicapi/whois/whois/google.com'\'' \\\r\n -H '\''accept: application/json'\'' \\\r\n -H '\''x-magicapi-key: api-key"
}
],
"temperature": 1,
"max_tokens": 256,
"top_p": 1,
"frequency_penalty": 0,
"stream": false
}'
```
{% endcode %}
#### Response
{% code overflow="wrap" %}
````bash
{
"id": "meta-llama/Meta-Llama-3-70B-Instruct-afc4c52e-ebab-4e8a-bebf-108ecf511758",
"object": "text_completion",
"created": 1718783159,
"model": "meta-llama/Meta-Llama-3-70B-Instruct",
"choices": [
{
"message": {
"role": "assistant",
"content": "Here is an example of an AWS Python Lambda function that calls the API with the given curl request and returns a response:\n```\nimport boto3\nimport json\nimport requests\n\ndef lambda_handler(event, context):\n # Extract the API key from the environment variables\n api_key = 'YOUR_API_KEY_HERE'\n\n # Extract the domain from the event\n domain = event['domain']\n\n # Construct the API URL\n url = f'https://api.magicapi.dev/api/v1/magicapi/whois/whois/{domain}'\n\n # Set the headers\n headers = {\n 'accept': 'application/json',\n 'x-magicapi-key': api_key\n }\n\n # Make the GET request\n response = requests.get(url, headers=headers)\n\n # Check if the response was successful\n if response.status_code == 200:\n # Return the response as JSON\n return {\n 'statusCode': 200,\n 'body': json.dumps(response.json())\n }\n else:\n # Return an error message\n return {\n 'statusCode': response.status_code,\n 'body': json.dumps({'error': 'API request failed'})\n }\n```\nHere's an explanation of the code:\n\n*",
"tool_calls": null,
"tool_call_id": null
},
"index": 0,
"finish_reason": "length",
"logprobs": null
}
],
"usage": {
"prompt_tokens": 105,
"completion_tokens": 256,
"total_tokens": 361
}
}
````
{% endcode %}
You can try this cheap and easy to use LLM API out here at
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.api.market/api-product-docs/bridgeml/meta-llama-3-70b-instruct.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
