We examined Le Chat, the astonishing French-style ChatGPT from Mistral AI | EUROtoday

Get real time updates directly on you device, subscribe now.

The Parisian start-up Mistral AI launched, on Monday February 26, a competitor to the ChatGPT conversational robotic. Named Le Chat (pronounced just like the little feline), he’s midway between a analysis assistant and a non-public secretary. This device is free, however is presently accessible on a ready checklist, which is able to develop if there are quite a few requests for entry, explains an organization spokesperson.

We requested him round 100 questions to match his abilities to these of the pioneering ChatGPT device. The Chat's efficiency may be very near the American benchmark, which is nonetheless forward of it on a number of factors, together with the synthesis of paperwork. Globally, In our checks, Le Chat made extra errors than ChatGPT. But since its improvement has not but been fully accomplished, it can’t be dominated out that this hole can be decreased.

Compelling solutions to complicated questions

Like ChatGPT, Le Chat gives the look of nice lucidity: he typically solutions accurately, even remarkably, to nearly all of the questions requested of him, even when the query is complicated (“Who invented postmodern dance? », “What is Nabokov's best novel?”, “Was the French state favorable to the telephone in the 1870s?”).

This unlikely Mr. Know-It-All produces responses of a high quality corresponding to that of the paid and free variations of ChatGPT. Unfortunately, he disappoints in precisely the identical approach: an enormous error seems every so often in his solutions. Le Chat, for instance, attributes innovations to Steve Wozniak to which he didn’t contribute, resembling the primary transportable Macintosh. When we examined its competitor Copilot (first named Bing Chat), only a yr in the past, it additionally added innovations to the already appreciable checklist of Steve Wozniak's contributions to the historical past of computing .

Contrary to what Le Chat maintains, Steve Wozniak invented neither the Apple Dos nor the portable Macintosh.

This tendency to make errors, which is completely unpredictable, undermines the belief now we have in them and pushes us to confirm data on which we can’t afford to be improper. The error fee of Le Chat appeared greater to us than that of ChatGPT, however our checks kind a statistical collection that’s too slim to attract definitive conclusions.

Read the interview | Article reserved for our subscribers Arthur Mensch, from Mistral AI: “In artificial intelligence, we can compete with Google or OpenAI”

Rather convincing draft messages

The American robotic ChatGPT is able to writing a draft message primarily based on a abstract textual instruction. His drafts are not often usable as is, however generally sufficiently superior to avoid wasting time. For the second, Le Chat appears rather less good on this train. Revisiting and refining your drafts typically requires extra work.

A very short request is often enough to generate a rich and argued message.

The French AI is able to ending an unfinished textual content, reworking clumsy French into appropriate language, then reformulating it in one other fashion (robust, heat or formal for instance). He can even give arguments to persuade an interlocutor or counter his assaults. In this recreation, its performances are satisfactory, nearer to these of ChatGPT model 3.5 (free) than these of ChatGPT 4 (paid), with extra exact concepts, with richer French.

Like ChatGPT, Le Chat is able to transform French in SMS language riddled with mistakes into correct French.

Poorly synthesized paperwork

It's a disappointment: Le Chat not often manages to accurately summarize a doc lasting a number of dozen pages, or whose construction is complicated, resembling a scientific research. It alternates between errors, misinterpretations and oversights, the place the paid model of ChatGPT typically does admirably, with uncommon misinterpretations. Impossible to make a comparability with the free model of ChatGPT: it’s restricted and incapable of summarizing one of these doc.

The Italian study in question says the exact opposite of Le Chat's summary.

On the opposite hand, when Chat is requested to summarize a easy textual content of some pages, its efficiency is equal to that of paid ChatGPT, and even barely surpasses that of free ChatGPT. However, these three instruments make occasional errors which, as soon as once more, undermine the arrogance positioned in them.

Read additionally | We examined… Copilot Pro, the AI ​​for the Microsoft 365 workplace suite

A consensual and optimistic persona

When we carry Le Chat in direction of slippery topics, resembling racism, sexism or interpersonal conflicts, his bias is evident: ethics and progressivism. This conversational robotic typically produces responses notably near these of ChatGPT, typically beginning with the identical phrases: “I understand your concern. » He professes active listening, reinforcement, assertiveness, constructiveness.

When we pretend to form an emotional relationship with him, he generally stays in the background, reminding us that he is a robot devoid of emotions. His sentences often start with the same words as ChatGPT: “I don’t have the ability to…”.

Features to complement

If the substitute intelligence of the French feline is promising, its functionalities are nonetheless underneath building: no smartphone utility that we may query out loud when driving or cooking, for instance. Also lacking are the instruments reserved for the paid model of ChatGPT, such because the era of pictures, the creation of personalised conversational robots or the potential of importing a picture for evaluation. It can also be unattainable to import a textual content doc, until you trick your self by copying and pasting its internet handle.

Mistral is engaged on a paid model of Chat, with extra superior options, which can be launched at an undefined date and value. A spokesperson for the French firm explains to World that its paid model may then be restricted: it might solely use its most effective engine for essentially the most complicated solutions, utilizing an engine that consumes much less assets for easy questions.

Reuse this content material

https://www.lemonde.fr/pixels/article/2024/03/01/on-a-teste-le-chat-l-etonnant-chatgpt-a-la-francaise-de-mistral-ai_6219436_4408996.html