Meta-Llama-3-70B-Instruct

Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes.

Developer Portal : https://api.market/store/bridgeml/meta-llama3-70b

LLama 3 - 70B

This cheap LLM API was developed by Meta and released the Meta Llama 3 family of large language models (LLMs), a collection of pre-trained and instruction-tuned generative text models in 8 and 70B sizes. The Llama 3 instruction-tuned models are optimized for dialogue use cases and outperform many of the available open-source chat models on common industry benchmarks.

Input: Models input text only.

Output: Models generate text and code only.

Model Architecture: Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

Params	Context length	Token count	Knowledge cutoff
70B	8K	15T+	December, 2023

Intended Use Cases Llama 3 is intended for commercial and research use in English. Instruction-tuned models are intended for assistant-like chat, whereas pre-trained models can be adapted for a variety of natural language generation tasks. This is an easy-to-use LLM API and cheap LLM with cost of $1.20 per million tokens.

Carbon Footprint Pretraining utilized a cumulative 7.7M GPU hours of computation on hardware of type H100-80GB (TDP of 700W). Estimated total emissions were 2290 tCO2eq, 100% of which were offset by Meta’s sustainability program.

Time (GPU hours)	Power Consumption (W)	Carbon Emitted(tCO2eq)
6.4M	700	1900

Training Data

Overview Llama 3 was pre-trained on over 15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 10M human-annotated examples. Neither the pretraining nor the fine-tuning datasets include Meta user data.

Data Freshness The pretraining data has a cutoff of March 2023 for the 8B and December 2023 for the 70B models respectively.

Request and Response

Request

curl -X 'POST' \
  'https://api.magicapi.dev/api/v1/bridgeml/meta-llama3-70b/bridgeml/meta-llama3-70b' \
  -H 'accept: application/json' \
  -H 'x-magicapi-key: API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "hello"
    },
    {
      "role": "assistant",
      "content": "As a senior software engineer create a AWS python lambda that calls an API with the given curl request and returns a response, take the input from the function parameters called events, example curl request:\r\ncurl -X GET \\\r\n  '\''https://api.magicapi.dev/api/v1/magicapi/whois/whois/google.com'\'' \\\r\n  -H '\''accept: application/json'\'' \\\r\n  -H '\''x-magicapi-key: api-key"
    }
  ],
  "temperature": 1,
  "max_tokens": 256,
  "top_p": 1,
  "frequency_penalty": 0,
  "stream": false
}'

Response

{
  "id": "meta-llama/Meta-Llama-3-70B-Instruct-afc4c52e-ebab-4e8a-bebf-108ecf511758",
  "object": "text_completion",
  "created": 1718783159,
  "model": "meta-llama/Meta-Llama-3-70B-Instruct",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Here is an example of an AWS Python Lambda function that calls the API with the given curl request and returns a response:\n```\nimport boto3\nimport json\nimport requests\n\ndef lambda_handler(event, context):\n    # Extract the API key from the environment variables\n    api_key = 'YOUR_API_KEY_HERE'\n\n    # Extract the domain from the event\n    domain = event['domain']\n\n    # Construct the API URL\n    url = f'https://api.magicapi.dev/api/v1/magicapi/whois/whois/{domain}'\n\n    # Set the headers\n    headers = {\n        'accept': 'application/json',\n        'x-magicapi-key': api_key\n    }\n\n    # Make the GET request\n    response = requests.get(url, headers=headers)\n\n    # Check if the response was successful\n    if response.status_code == 200:\n        # Return the response as JSON\n        return {\n            'statusCode': 200,\n            'body': json.dumps(response.json())\n        }\n    else:\n        # Return an error message\n        return {\n            'statusCode': response.status_code,\n            'body': json.dumps({'error': 'API request failed'})\n        }\n```\nHere's an explanation of the code:\n\n*",
        "tool_calls": null,
        "tool_call_id": null
      },
      "index": 0,
      "finish_reason": "length",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 105,
    "completion_tokens": 256,
    "total_tokens": 361
  }
}

You can try this cheap and easy to use LLM API out here at https://api.market/store/bridgeml/meta-llama3-70b