API.Market
Go to API.market
  • Welcome to API.market
  • What are API Products?
  • How to subscribe to a SaaS API Product?
  • Managing Subscriptions
  • Analytics & Logs
  • How can I cancel my Subscription?
  • How do I add payment details?
  • How does API.market charges me?
  • Error Codes
  • API.market Usage API Documentation
  • Seller Docs
    • API Seller Console
    • What is an API Product?
    • What is a Pricing Plan
    • Importing an API Source
    • Creating a Product using the Wizard
    • Testing Your APIs & Products
    • Analytics & Logs
    • Custom Usage
    • Overriding Custom Usage on Result Retrieval
  • FUNDAMENTALS
    • Convert Postman Collection to OpenAPI Yaml
    • Create OpenAPI spec using ChatGPT
  • About Us
  • API Product Docs
    • MagicAPI
      • Screenshot API
      • Domain Availability Checker API
      • WhoIS API
      • PDF Conversion API
      • Image Upscale API
      • DNS Checker API
      • Ageify API
      • Image Restoration API
      • Toon Me API
      • Coding Assistant
      • 🎭 FaceSwap API: Instantaneous replaces face with one another
      • 🏞️ Image Upload API
      • Deblurer API
      • Hair Changer API
      • 🤳🏻🤖AI Qr Code Generator API
      • Whisper API
      • Image Colorizer API
      • OpenJourney API
      • Object Remover API
      • Image Captioner API
      • Object Detector API
      • NSFW API
      • Crunchbase API
      • Pipfeed's Extract API Developer Documentation
      • Migrating from Capix FaceSwap API to magicapi/faceswap-capix API
    • BridgeML
      • Meta-Llama-3-8B-Instruct
      • Meta-Llama-3-70B-Instruct
      • Mistral-7B-Instruct-v0.1
      • Mixtral-8x22B-Instruct-v0.1
      • Meta-Llama-2-7b
      • Meta-Llama-2-13b
      • Meta-Llama-2-70b
      • Gemma-7b-it
      • NeuralHermes-2.5-Mistral-7B
      • BAAI/bge-large-en-v1.5
      • CodeLlama-70b-Instruct-hf
      • 🤖🧗Text-to-Image API
      • 📝🎧 Text to Audio API
    • Capix AI
      • FaceSwap Image and Video Face Swap API
      • MakeUp
      • Photolab.me
      • AI Picture Colorizer
      • AI Picture Upscaler
      • AI Background Remover
      • Object Remover
      • TTS Universal
      • Home GPT
      • AI & Plagiarism Checker
      • AI Story Generator
      • AI Essay Generator
      • Book Title Generator
    • Trueway
      • ⛕ 🗺️ Trueway Routing API
      • 🌐📍Trueway Geocoding API: Forward and Reverse Geocoding
      • 🛤️ ⏱️Trueway Matrix API: Travel Distance and Time
      • 🏛️ Trueway Places API
    • AILabTools
      • Cartoon-Yourself
    • SharpAPI
      • 📄 AI-Powered Resume/CV Parsing API
      • 🛩️ Airports Database & Flight Duration API
    • Text to Speech
      • Turn your text into Magical-sounding Audio
Powered by GitBook
On this page
  • NeuralHermes 2.5 - Mistral 7B
  • Quantized models
  • Results
  • Training hyperparameters
  • Request and Response
  1. API Product Docs
  2. BridgeML

NeuralHermes-2.5-Mistral-7B

NeuralHermes-2.5-Mistral-7B: Unleash unparalleled AI capabilities with Mistral's 7B parameters for advanced natural language processing.

PreviousGemma-7b-itNextBAAI/bge-large-en-v1.5

Last updated 12 months ago

Developer Portal : https://api.market/store/bridgeml/mlabonne

NeuralHermes 2.5 - Mistral 7B

NeuralHermes is based on the teknium/OpenHermes-2.5-Mistral-7B model that has been further fine-tuned with Direct Preference Optimization (DPO) using the mlabonne/chatml_dpo_pairs dataset. It surpasses the original model on most benchmarks (see results).

It is directly inspired by the RLHF process described by Intel/neural-chat-7b-v3-1's authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.

The code to train this model is available on Google Colab and GitHub. It required an A100 GPU for about an hour.

Quantized models

  • GGUF: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF

  • AWQ: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-AWQ

  • GPTQ: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GPTQ

  • EXL2:

    • 3.0bpw: https://huggingface.co/LoneStriker/NeuralHermes-2.5-Mistral-7B-3.0bpw-h6-exl2

    • 4.0bpw: https://huggingface.co/LoneStriker/NeuralHermes-2.5-Mistral-7B-4.0bpw-h6-exl2

    • 5.0bpw: https://huggingface.co/LoneStriker/NeuralHermes-2.5-Mistral-7B-5.0bpw-h6-exl2

    • 6.0bpw: https://huggingface.co/LoneStriker/NeuralHermes-2.5-Mistral-7B-6.0bpw-h6-exl2

    • 8.0bpw: https://huggingface.co/LoneStriker/NeuralHermes-2.5-Mistral-7B-8.0bpw-h8-exl2

Results

Update: NeuralHermes-2.5 became the best Hermes-based model on the Open LLM leaderboard and one of the very best 7b models. 🎉

Teknium (author of OpenHermes-2.5-Mistral-7B) benchmarked the model (see his tweet).

Results are improved on every benchmark: AGIEval (from 43.07% to 43.62%), GPT4All (from 73.12% to 73.25%), and TruthfulQA.

AGIEval

GPT4All

TruthfulQA

You can check the Weights & Biases project here.

Training hyperparameters

LoRA:

  • r=16

  • lora_alpha=16

  • lora_dropout=0.05

  • bias="none"

  • task_type="CAUSAL_LM"

  • target_modules=['k_proj', 'gate_proj', 'v_proj', 'up_proj', 'q_proj', 'o_proj', 'down_proj']

Training arguments:

  • per_device_train_batch_size=4

  • gradient_accumulation_steps=4

  • gradient_checkpointing=True

  • learning_rate=5e-5

  • lr_scheduler_type="cosine"

  • max_steps=200

  • optim="paged_adamw_32bit"

  • warmup_steps=100

DPOTrainer:

  • beta=0.1

  • max_prompt_length=1024

  • max_length=1536

Source

Request and Response

Request

curl -X 'POST' \
  'https://api.magicapi.dev/api/v1/bridgeml/mlabonne/bridgeml/mlabonne' \
  -H 'accept: application/json' \
  -H 'x-magicapi-key: API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "hello"
    },
    {
      "role": "assistant",
      "content": "Hello, how can you help me?"
    }
  ],
  "temperature": 1,
  "max_tokens": 256,
  "top_p": 1,
  "frequency_penalty": 0,
  "stream": false
}'

Response

{
  "id": "mlabonne/NeuralHermes-2.5-Mistral-7B-eab3ca77-e1d0-41cb-b43e-e8195af77dc7",
  "object": "text_completion",
  "created": 1718905083,
  "model": "mlabonne/NeuralHermes-2.5-Mistral-7B",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "It seems like you have a question or need assistance with something. Please feel free to provide more information or context so I can do my best to help you.",
        "tool_calls": null,
        "tool_call_id": null
      },
      "index": 0,
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 76,
    "completion_tokens": 33,
    "total_tokens": 109
  }
}

You can use this easy to use and cheap LLM Api here at https://api.market/store/bridgeml/mlabonne