Dear all,

Yes, same experience here.

I’m working on a bibliography about ‘European package leaflets’.  http://graphicdesign-research.com/PackageLeaflets/bibliography.html 

I asked ChatGTP to make a bibliography: it gave the answers below. None of these articles or references exist: the information  is all mangled. 

Annoying thing is that they look genuine and it takes time to check. Waste of time.

Alas, it is still essential to find the original publication, read it carefully, and check the quotations and references. Even without ChatGTP many inaccuracies in published peer reviewed papers will come to light ...

Kind regards,
Karel.
[log in to unmask]

>>>
My question: Can you make a bibliography of articles about european package leaflets?
ChatGPT answersed: Certainly! Here are some articles about European package leaflets:

"Patient information leaflets for medicines: using consumer testing to determine the most effective design." European Medicines Agency. Available at: https://www.ema.europa.eu/en/documents/presentation/patient-information-leaflets-medicines-using-consumer-testing-determine-most-effective-design_en.pdf <https://www.ema.europa.eu/en/documents/presentation/patient-information-leaflets-medicines-using-consumer-testing-determine-most-effective-design_en.pdf>
"The impact of the new European Union patient information leaflet format on understanding and adherence." by M. C. Gagnon et al. Health Expectations, vol. 18, no. 6, 2015, pp. 3122-3133.

"Package leaflets for medicines: guidance for industry." European Medicines Agency. Available at: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/package-leaflets-medicines-guidance-industry_en.pdf <https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/package-leaflets-medicines-guidance-industry_en.pdf>
"Improving European package leaflets for medicines: results from the PEL project." by B. A. Williams et al. Health Policy, vol. 116, no. 1, 2014, pp. 34-43.

"The readability of European pharmaceutical package leaflets." by C. R. Horne et al. The Annals of Pharmacotherapy, vol. 39, no. 6, 2005, pp. 1008-1013.

"How to make package leaflets more useful." by J. J. de Gier et al. European Journal of Clinical Pharmacology, vol. 64, no. 3, 2008, pp. 259-267.

"Development of a standard set of patient-relevant outcomes for the evaluation of new medicines: the European Medicines Agency's perspective." by H. J. Schünemann et al. Value in Health, vol. 22, no. 2, 2019, pp. 128-136.

"European package leaflets: a comparative study." by R. J. Wensing-Kruger et al. Pharmacoepidemiology and Drug Safety, vol. 11, no. 6, 2002, pp. 429-437.

"Patient information leaflets (PILs): a review of past, present and future use." by R. Krska et al. Pharmacy World and Science, vol. 29, no. 5, 2007, pp. 491-497.

I hope this helps!

>>>>

> On 27 Mar 2023, at 08:15, Ken Friedman <[log in to unmask]> wrote:
> 
> Dear All,
> 
> This morning, I received a note from Michael Lissack, a professor at Tongji and an editor of She Ji. He has been working on an article, and he decided to see what ChatGPT could help him find. He wrote that after looking for relevant articles and citations, he decided to ask “What the researcher Michael Lissack had to say on the topic.”
> 
> He had a frightening experience. He wrote that "It came up with some good stuff BUT also quotes from papers i never wrote with co-authors I never wrote with complete with full citations and doi numbers.”
> 
> He realised that he must double and triple check every potential citation it gave him.
> 
> He posted a short essay to LinkedIn. You can read it below. It is worth reading. Journal reviewers and conference reviewers must be even more careful than they have been in the past to ensure that citations are genuine and accurate.
> 
> Yours,
> 
> Ken Friedman 
> 
> --
> 
> ChatGPT. Danger Academic Researchers Danger
> 
> This is a warning to those thinking of (or who have) used ChatGPT to assist in doing serious research ... be very careful ChatGPT makes up quotes and references. Just because it lists a full citation... does not mean that citation is real.
> 
> After using the system for roughly 90 minutes, and getting back what seemed to be a treasure trove of quotes and references to use for my next article, I decided to ask the system "what has the researcher Michael Lissack said about this concept?" Back came 10 quotes. 3 I recognized. 7 surprised me -- especially the ones from articles I had supposedly written with co-authors I had never co-authored with. My favorite was a supposed quote from a paper ChatGPT claimed I co-authored with Norbert Weiner in 2019. My ability to pull off miracles is just not that great.
> 
> If you use ChatGPT for research, please please double and triple check quotes and references. I suggest putting the reference as written in Google Scholar and just rejecting that which fails to have an exact match. You could do the same with the text of a quote and see what happens.
> 
> Attention Journal Editors: this is NOT something a peer reviewer will catch (unless of course they are listed as authoring an unknown paper). There are algorothmic solutions for reference checking ...thanks to ChatGPT"s "hallucinations" (yup OpenAI gave this phenomenon a name) they MUST be used on every article from now on.
> 
> The hallucinized "quotes" are sometime better than the real thing ... more coherent and resonant... and thus all the more dangerous.
> 
> i was hoping ChatGPT could locate the unfinished paper I supposedly worked on with Kafka, but it remains lost in The Castle
> 
> Warning has been given #research #google #references #chatgpt #openai
> 
> *********************
> 
> ChatGPT reacted:
> 
> To check for fake quotes and references supplied by ChatGPT, a researcher can use the following methods:
> 
> Check the reference against a reliable database: Researchers can use Google Scholar, JSTOR, or other reputable databases to check if the reference exists and is attributed to the correct author.
> 
> Verify the quote: Researchers can copy and paste the quote into a search engine to see if it appears in any credible sources. They can also search for the quote in quotation marks to see if it is attributed to the correct author.
> 
> Check for inconsistencies: Researchers can check for inconsistencies in the quote, such as a different writing style or language from the author's usual style. They can also check for any errors in the citation or reference.
> 
> Use human judgment: Ultimately, researchers should use their judgment and knowledge of the field to determine if the quote or reference is authentic and relevant to their research.
> 
> It is essential to be vigilant and use multiple sources to verify the accuracy of any information obtained from ChatGPT or any other AI language model.
> 
> *************
> 
> In simple English: don't trust ChatGPT
> 
> 
> -----------------------------------------------------------------
> PhD-Design mailing list  <[log in to unmask]>
> Discussion of PhD studies and related research in Design
> Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
> -----------------------------------------------------------------



-----------------------------------------------------------------
PhD-Design mailing list  <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------