Hi all, This is my first message here, and I couldn't resist the topic. I'd
tend to side with Nuno - My technical expertise is limited here but the
consensus seems that at least for Large Language Models, we are already
hitting a plateau and that more focused models with better training might
be the way forward - https://dl.acm.org/doi/10.1145/3442188.3445922 this
article (Stochastic parrots, widely cited) is a bit old now but touches
upon some of the issues with the current "bigger is better" approach to
LLMs. I also second Gabriele with the reference to the excellent work of
Timnit Gebru and colleagues like Dr. Joy Buolamwini do concerning immediate
harms like biases.
I'm wondering if the existential worry in design is exacerbated by the
current trend of packaging our activities into a marketable 'process' -
design thinking, rather than acknowledging the role of the practitioner.
When design is seen as a process with inputs and outputs, it risks reducing
what we do to something akin to a machine, thus making it feel more
susceptible to the threat of being replaced by AI. This perspective does a
disservice to design creatives, who in my opinion are not just wielders of
tools and processes. A designer is an embedded person, with motivations,
past lived experiences, and opinions (a person who brings these elements,
consciously or not, to their practice). These are aspects that current AI
models, regardless of their sophistication in terms of KTokens, MTokens, or
TTokens, do not possess (although they can be simulated if asked), and
would require a fundamental shift in their design.
I think we could reference Heidegger's concept of Dasein, or
'being-in-the-world' - the existential nature of human experience is absent
in AI. Our work as designers is a reflection of our way of being and
interacting with the world, within our cultural, historical, and personal
contexts. Perhaps this could be an argument for a return of a sort of
"authorship" of design?
--- Gaston Welisch Researcher + PhD Applicant Glasgow School of Art
https://gaston.pro/
---
On Mon, 20 Nov 2023 at 13:34, Nuno Jardim Nunes <
[log in to unmask]> wrote:
> I would be careful in thinking that token size could follow the same
> exponential growth of Moore’s law, leading to an exponential increase in
> the quality of LLMs. It will undoubtedly increase computational effort, but
> that’s the only evidence we have.
>
> So far, we know that while increasing the number of tokens generally
> improves the quality of LLM outputs, there are practical and technical
> limits. Beyond a certain point, further increases in token count may not
> yield proportionate improvements in quality, and other factors like
> training data quality and computational efficiency become increasingly
> important.
>
> Technical improvements (and simple market competition) will decrease the
> cost but increase the number of tokens, but I think we will see LLMs
> hitting limitations sooner than we expect:
>
> Here is an interesting technical read in one angle of this problem:
> https://doi.org/10.48550/arXiv.2305.18654
> --
> Nuno Jardim Nunes
> Professor HCI - Técnico - U. Lisbon
> http://nunojnunes.me
> President | Interactive Technologies Institute | LARSyS |
> http://iti.larsys.pt
>
> > On 20 Nov 2023, at 08:49, Ferri, Gabriele <
> [log in to unmask]> wrote:
> >
> > Hi Andre, thanks for the information.
> >
> > I suppose it's a matter of understanding what do we mean by "design,"
> which is a topic that - whenever it's raised - causes heated intellectual
> debates in this list and elsewhere 😉 Perhaps it's not the best move to
> dive into that rabbit hole again. Or, at least, I don't have the energy for
> another round right now.
> >
> >
> >
> > On the other hand, I'd like to take this opportunity to mention the very
> interesting work by Timnit Gebru and colleagues unpacking the immediate
> harms caused by LLMs and AI (whatever this umbrella term actually means).
> They argue against the tendency to fixate on longer-term risks (AGI etc)
> and for the need to attend immediate harms instead (energy/water use, labor
> exploitation, etc). Their foundational paper is
> https://doi.org/10.1145/3442188.3445922, this is another interesting
> entry point https://doi.org/10.48550/arXiv.2210.02667, and I've written a
> commentary/analysis recently https://doi.org/10.1145/3571884.3603751
> >
> >
> >
> > Cheers
> >
> > G:
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: PhD-Design <[log in to unmask]> On Behalf Of andré neves
> > Sent: Sunday, November 19, 2023 7:10 PM
> > To: [log in to unmask]
> > Subject: Human, AI and Design Activities
> >
> >
> >
> > TeraTokens
> >
> >
> >
> > ChatGPT 4.0 currently operates within the realm of KTokens, with a
> context window of 128KTokens, comparable to the leap experienced with the
> 16KBytes
> >
> > ZX81 in the 1980s.
> >
> >
> >
> > What transformations are expected when LLMs advance to using TTokens
> instead of KTokens?
> >
> >
> >
> > In the design field, significant activities have traditionally relied on
> human capacity to analyze situations and propose creative solutions.
> >
> > However, AI, with its rapid cognitive abilities, is poised to
> revolutionize these design activities.
> >
> >
> >
> > Short Term (KTokens): In tasks where basic design principles are
> applied, AI can automate and streamline processes, potentially reducing the
> need for human intervention in simpler design tasks.
> >
> >
> >
> > Medium Term (MTokens): In more complex areas where understanding user
> needs and environmental factors is crucial, AI might begin to significantly
> contribute, bringing efficiency and data-driven insights.
> >
> >
> >
> > Long Term (GTokens): In highly creative fields where innovative thinking
> and emotional intelligence are key, AI could offer advanced support and
> collaboration, enhancing the creative process with new perspectives and
> tools.
> >
> >
> >
> > This shift in design activities poses ethical and social challenges,
> necessitating a reevaluation of the role of human designers and the
> integration of AI in a manner that enhances rather than replaces human
> creativity.
> >
> >
> >
> > Are we seriously and impartially discussing this topic in our groups,
> whether in academia or the industry?
> >
> >
> >
> >
> >
> > -----------------------------------------------------------------
> >
> > PhD-Design mailing list <[log in to unmask]<mailto:
> [log in to unmask]>> Discussion of PhD studies and related
> research in Design Subscribe or Unsubscribe at
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.jiscmail.ac.uk%2Fphd-design&data=05%7C01%7C%7C72d9e5c450ab48c1ba7c08dbe92ab6bc%7Ccc7df24760ce4a0f9d75704cf60efc64%7C0%7C0%7C638360141890091906%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BupOhTeZkd32oNB7XqEqrq43Zx6YJVjgV3qKVUZbUis%3D&reserved=0
> <https://www.jiscmail.ac.uk/phd-design>
> >
> > -----------------------------------------------------------------
> >
> >
> > -----------------------------------------------------------------
> > PhD-Design mailing list <[log in to unmask]>
> > Discussion of PhD studies and related research in Design
> > Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
> > -----------------------------------------------------------------
>
>
>
> -----------------------------------------------------------------
> PhD-Design mailing list <[log in to unmask]>
> Discussion of PhD studies and related research in Design
> Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
> -----------------------------------------------------------------
>
-----------------------------------------------------------------
PhD-Design mailing list <[log in to unmask]>
Discussion of PhD studies and related research in Design
Subscribe or Unsubscribe at https://www.jiscmail.ac.uk/phd-design
-----------------------------------------------------------------
|