llm-gpt4all

为LLM添加对GPT4All模型集合的支持的插件。

安装

在LLM相同的环境中安装此插件。

llm install llm-gpt4all

安装插件后，您将看到一个新列表，其中包含可用的模型，如下所示

llm models list

输出将包含类似以下内容

gpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM (installed)
gpt4all: nous-hermes-llama2-13b - Hermes, 6.86GB download, needs 16GB RAM (installed)
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: replit-code-v1_5-3b-q4_0 - Replit, 1.74GB download, needs 4GB RAM
gpt4all: mpt-7b-chat-merges-q4_0 - MPT Chat, 3.54GB download, needs 8GB RAM
gpt4all: rift-coder-v0-7b-q4_0 - Rift coder, 3.56GB download, needs 8GB RAM
gpt4all: em_german_mistral_v01 - EM German Mistral, 3.83GB download, needs 8GB RAM
gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM
gpt4all: mistral-7b-openorca - Mistral OpenOrca, 3.83GB download, needs 8GB RAM
gpt4all: gpt4all-falcon-q4_0 - GPT4All Falcon, 3.92GB download, needs 8GB RAM
gpt4all: gpt4all-13b-snoozy-q4_0 - Snoozy, 6.86GB download, needs 16GB RAM
gpt4all: wizardlm-13b-v1 - Wizard v1.2, 6.86GB download, needs 16GB RAM
gpt4all: starcoder-q4_0 - Starcoder, 8.37GB download, needs 4GB RAM

有关这些模型的更多详细信息，可以在此Observable笔记本中找到。

使用方法

模型安装和提示

您可以使用在llm models list输出中显示的名称执行模型。第一次尝试运行它时，将下载模型文件。

llm -m orca-mini-3b-gguf2-q4_0 '3 names for a pet cow'

第一次运行此命令时，您将看到模型文件下载的进度条

 31%|█████████▋                        | 1.16G/3.79G [00:26<01:02, 42.0MiB/s]

在随后的使用中，模型输出将立即显示。

请注意，模型将下载到~/.cache/gpt4all。

模型选项

运行llm models --options以获取可用模型选项的列表，其中应包括

gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM (installed)
  max_tokens: int
    The maximum number of tokens to generate.
  temp: float
    The model temperature. Larger values increase creativity but decrease
    factuality.
  top_k: int
    Randomly sample from the top_k most likely tokens at each generation
    step. Set this to 1 for greedy decoding.
  top_p: float
    Randomly sample at each generation step from the top most likely
    tokens whose probabilities add up to top_p.
  repeat_penalty: float
    Penalize the model for repetition. Higher values result in less
    repetition.
  repeat_last_n: int
    How far in the models generation history to apply the repeat penalty.
  n_batch: int
    Number of prompt tokens processed in parallel. Larger values decrease
    latency but increase resource requirements.

使用方法如下

llm -m mistral-7b-instruct-v0 -o max_tokens 2 'hi'

Hello!

聊天

要与模型聊天，避免每次消息都需要将其加载到内存中，请使用llm chat

llm chat -m orca-mini-3b-gguf2-q4_0

Chatting with orca-mini-3b-gguf2-q4_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> hi
 Hello! How can I assist you today?
> three jokes about a pelican and a chicken who are friends
 Sure, here are three jokes about a pelican and a chicken who are friends:

1. Why did the pelican cross the road? To get to the other side where the chicken was waiting for him!
2. What do you call a group of chickens playing basketball? A flock of feathers!
3. Why did the chicken invite the pelican over for dinner? Because it had nothing else to eat and needed some extra protein in its diet!

删除模型

要删除已下载的模型，请从~/.cache/gpt4all中删除.gguf文件。

开发

要本地设置此插件，首先检出代码。然后创建一个新的虚拟环境

cd llm-gpt4all
python3 -m venv venv
source venv/bin/activate

现在安装依赖关系和测试依赖关系

pip install -e '.[test]'

要运行测试

pytest