This talk by Georgia Ward Dyer from Royal College of Art on "O Time thy pyramids" was presented at the Calligraphic Traces event on 31st July at Thoughtworks as part of the Creative AI meetup. The upload consists of slides followed by Georgia's notes from the talk.

1. Georgia Ward Dyer
• creative research
• machine learning ∩ philosophy
gfwd23@gmail.com / @GeorgiaWD

2. O Time thy pyramids

3. padlock

4. electric guitar

5. cowboy hat

6. street sign

7. fountain

8. scorpion

11. scorpion

13. scorpion

14. 口
mouth

15. 口
mouth
唱
to sing

16. 口
mouth
唱
to sing
叫
to call

17. 口
mouth
唱
to sing
叫
to call
吃
to eat

19. 书
book

20. 书 B
book

22. gfwd23@gmail.com / @GeorgiaWD

23. Notes from the talk
• Recent work in the field of machine vision can have applications
(and implications) for philosophical questions of meaning, and
ontology. For example fooling images (see Anh Nguyen, Jason
Yosinski, and Jeff Clune. "Deep neural networks are easily fooled:
High confidence predictions for unrecognizable images." 2015
IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2015. doi:10.1109/cvpr. 2015.7298640.), Quick, Draw!
dataset.
• In this talk I present my own work O Time thy pyramids (title is a
quote from Borges’ The Library of Babel). A series of ink
drawings, with accompanying prints. The ink drawings are
abstract (more on process later) in the tradition of Surrealists’
‘automatic drawing’, i.e. mark-making with no preconceived
intention about the subject of the drawing. Drawings are submitted
to an image recognition model (convnet) which identifies what it
thinks the image is of, giving higher or lower scores to the labels

24. • Taking one of these labels, for example ‘scorpion’, the
accompanying print shows the particular parts of the original
drawing which prompted that label (‘scorpion-y’ parts). [NB I did
not always choose the highest class probability/‘top’ label.]
• Brief introduction to Deep Visualization Toolbox: Credit to
Yosinski et al (‘Understanding Neural Networks Through Deep
Visualization’ 2015). Tool which gives you visualisation of what is
happening throughout the layers. I used it to obtain the
deconvolutional visualisation - which visualises which pixels in
the input image caused a specific neuron to activate. This
visualisation is what I used for the prints.

25. • Why is this interesting? Philosophical ideas about visual representation
and perception.
• How is an image of something like the thing itself? What does a tree
have in common with a drawing of a tree? They ‘look similar’. But (how)
can we deconstruct exactly what ‘looking similar’ means..? ‘Tertium
comparationis’ = ‘third part’, i.e. the quality that two things being
compared have in common. Is it possible to isolate the ‘tertium
comparationis’ between a thing and its image? Whatever that looks
like, could we think of it as the ‘essence’ of that thing - what makes it
that and not something else, that which gives it its identity? [NB this is
of course an essentialist view, not and not necessarily one which I think
is correct].
• This is what the prints show. A visualisation of what made the model
say ‘scorpion’. A practical answer to the theoretical question, ‘what is
the third part between this image and a scorpion’? In fact it is only that
third part - there is a connection to information theory (images are
information - pixel data): insofar as this image carries meaning, then for
the message ‘scorpion’, here is a visualisation of the minimum
information - an efficient scorpion?!
• [NB not scientifically rigorous - artistic licence but also process. E.g.
scanning instead of photographing to turn the drawings into digital files
affected output results]

26. • Of course this is visual representation as per machine intelligence,
not human. The model’s notion of what is ‘scorpion-y’ is a machinic
not a human one, computed by assimilating across a collection of
different images in the training set which were all labelled as
‘scorpion’. However what is interesting is human viewers look at the
two images side by side and immediately try to ‘see’ the label in the
right-hand image (especially in exhibition where careful placement of
the caption has significant impact). There is reflexivity: human-made
image, then interpreted by machine, then understood/meaning read in
by human viewers. Reflexivity is fitting - cf. Kate Storrs’ Creative AI
talk about DNNs as models of the brain.
• Why is it interesting to consider minimum scorpion/‘essence of
scorpion’? Identity and meaning. The real ‘zero-redundancy’ scorpion
would be a pixel away from being something else completely - if every
bit of information in the image is necessary for it to be classified as a
scorpion, then you couldn’t change any part of it without suddenly
shifting it’s identity. Paring back an image so that every part of it is
crucial to its ‘meaning’ is interesting. Borderlines - how far can you
take x until it’s something else, or until it breaks - this is how you
discover the shape of something; its definition and its identity.

27. • Drawings process & calligraphic connection: Borderline between
representational and abstract.
• Abstract drawings (no subject that I had in mind to represent) yet
I’m asking them (absurdly) to carry meaning in a representational
sense. Aesthetic - intentionally want them read as ‘calligraphic’,
sharing visual identity with the written letter (glyph). They are not
letters or characters! But they look like they could be. Drawing
process: I didn’t consciously try to make drawings that looked
letter-like, but focused more on performance of the brushstroke,
on gesture of mark-making. Cf Daniel Berio’s talk re: modelling
that gesture as a critical part of making convincing, natural-
looking glyphs. Cf William C Watt, linguist and cognitive
semiotician who developed gesture theory of the origins of letter
shapes.
• I made these ‘calligraphic’ in aesthetic in order to invoke the
relationship between meaning and glyph - not a straightforward
relationship. Some letter systems have roots in representation (as
in visual similarity) which are still visible today, e.g. chinese
radical for mouth - the root for sing, call, eat, but it’s not a
straightforward representational story.

28. • Chinese characters - meeting visitor at exhibition who drew the
Chinese characters which she thought my drawings strikingly
close to. Drawing which became ‘cowboy hat’ looks like shū
(book) but also letter ‘B’. Viewers’ personal experience (visual
references) affects what they perceive in the abstract drawing.
Many alphabet systems are not based on visual similarity, but
function as symbols (‘represent’ their meaning but not in a
visually similar sense). How do we read drawings as meaningful,
in particular in this instance whether we’re reading them as
representationally/ symbolically meaningful? Borderline between
representational and ‘fictionally symbolic’ (?). When is the letter a
not the letter a? Perhaps when it’s not recognisable. But humans
are so excellent at pattern recognition.
• Why was I successful in invoking glyph reference through
drawings? There must be certain visual rules that everyone
subconsciously knows about what a glyph - in any alphabet -
looks like, a certain system of angles and spacing, complexity of
composition. Cf Daniel’s work again. Exciting to explore these
sorts of conceptual questions practically through AI.