Script to interact with open source LLM

Support and Helpdesk
1

Dear all

I was trying to use open source LLMs (Llama3, Mistral etc) through Groq API. Groq supports POST for chat completion. In one column I'm having queries, and want to add another column (based on this) through chat completion API by following the example Groq has given (GroqCloud). This script is generating Error in OpenRefine but works fine in command prompt (python3 test3-groq.py). The error (Python/Jython) in OpenRefine is : Error: SyntaxError: mismatched input ':' expecting NEWLINE (_types.py, line 91).

The script:

from groq import Groq
client = Groq(
    api_key='my-api-key-is-here',
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "system",
            "content": "you are a helpful assistant."
        },
        {
            "role": "user",
            "content": value,
        }
    ],
    model="llama3-70b-8192",
)

print(chat_completion.choices[0].message.content)

What am I missing?

Regards

Parthasarathi

1 Like
2

Expressions in Jython/Python must have a return statement.

Change that print statement to a return statement.

return chat_completion.choices[0].message.content

See Docs: Jython & Clojure | OpenRefine

1 Like
3

Thanks Thad.

Applied this but the same error: Error: SyntaxError: mismatched input ':' expecting NEWLINE (_types.py, line 91)

4

This script is generating Error in OpenRefine but works fine in command prompt (python3 test3-groq.py).

Does it work with Python 2.7? That's the version that OpenRefine bundles.

Tom

5

Copilot says:

The Python script you’ve shared uses the groq library, which provides convenient access to the Groq REST API. However, it seems that the code snippet you posted is not directly related to Python 2.7 compatibility. Let me address that aspect for you.

Python 2.7 Compatibility:

If you have any further questions or need assistance with a different topic, feel free to ask! :blush:

6

Thanks to both of you for pointing the root cause for this issue. Now I understand why colleagues are using modules like urlbib, request etc for managing POST responses. I will try it in that way.

Regards

Parthasarathi