As you’ll know, Chat GPT is a chatbot, much hyped in the media. It will write a good and very convincing paragraph or two, but, be wary — the bot will also make stuff up if it doesn’t find an answer in its database. This new bot is not connected to the internet and doesn’t know much after 2021. Don’t get me wrong, it has its uses, but we are the beta testers!
Testing the limits
Here’s an example. I was looking for material for an article on Cassowaries (i.e., very big flightless birds found in Far North Queensland, where I live) and I wanted to test the limits of Chat GPT.
I am pretty familiar with Cassowaries, which are rated, unfairly I think, as one of the most dangerous bird species in the world.
ChatGPT gave me a good starting paragraph. So I asked for references — yes, easily done — “here you are”. All so convincing! Smith, A.B. and colleagues did this, and Jones C.D. did that and came up with this (names are changed to protect the misquoted, with apologies to A. Smith and C. Jones, wherever they may be!). It even threw in Sir David Attenborough.
I requested the references for Smith and Jones and was given the citations, which included all the expected details (i.e., good scientific journals, page numbers, DOIs — Digital Object Identifiers, those strings of numbers, letters and symbols used to permanently identify an article or document, and link to it on the web). So I hit the links. No Smith or Jones and the articles were on butterflies and seagulls.
Back to ChatGTP. Surprisingly, it apologised profusely and gave me another convincing reference — same format… similar result, with no sign of our two intrepid scientists or the bird in question. So I went to the journals themselves and searched on Smith and Jones — neither had ever been published in either journal. We went through a couple of rounds of this, with different DOIs and journals. Back to the bot, once again the apology, this time with the admission that Smith, in this case, hadn’t actually published the work. Not that the bot had simply made them up!
As it turned out, on further digging, neither Smith, A.B. nor Jones, C.D. had ever done any work on Cassowaries. Those authors were entirely fictitious, which makes me wonder about the rules governing the Chat GTP’s responses, and about its sources of information, given the accuracy of the bot’s first reply.
So, be wary — we are beta-testing ChatGPT. Let’s hope the bot eventually lives up to the hype!