Shaping, Enacting, Deciding: Data, AI and Educational Judgement

When I agreed with Luci Pangrazio to give a seminar at REDI, at Deakin, I knew it came with quite a lot of responsibility: because of the respect I have for her work, because I was going to Melbourne, and because I really didn’t want —I never do— to repeat myself too much… So, as the date got closer, I started to have a problem that, by that point in my stay, was quite real: I had already talked about a lot of things… and I don’t have THAT much to say that I can keep talking without ever repeating myself… perhaps they don’t actually need me to tell them anything… you know, a textbook case of impostor syndrome, combined with a dose of realism.

I had already given the four Educational Optics seminars at UTS, I had been to Griffith and, one way or another, I had already talked about PLEs, agency, digital teaching competence, professional judgement, assessment, datafication, AI as a prism, and my own experiments with AI in my teaching. And, on top of that, I was going to REDI, which was hardly the place to go and explain that data are not neutral, that platforms shape what we do, or that technology is not simply a tool, because Luci and many of the people working there have been working on precisely some of those questions for years.

So I knew quite clearly what I didn’t want to do: another Educational Optics, or to take the bits I liked best from the previous seminars, rearrange them and pretend I had a new talk. I wanted to use the opportunity to try to go a little further than what I had already talked about during these months, with the small inconvenience that I didn’t really know where.

But that is the great thing about preparing these things: you get to think… For me, preparing a talk properly —you can’t always do it, or don’t always have the conditions to do it— gives you time and a very good excuse to think, and without all the constraints of a paper.

The title, Shaping, Enacting, Deciding: Data, AI and Educational Judgement, had appeared quite a long time before the actual talk —when I wrote the abstract, which was already difficult enough— and it contained three verbs that interested me and a still rather messy intuition: when I looked at data and AI from the perspective of the things I have been working on for years, I had the feeling that they were running through, all at once, problems that I had previously approached separately, and that perhaps putting them together would allow me to think about something that I still didn’t quite know how to formulate.

Obviously, I started by positioning myself again, where I am and where I come from… because I am not THAT important that everyone should already know where I come from or what I think, and if I want them to think along with me, I need to give them some coordinates. I didn’t want to construct a retrospective story, but rather to show that work that emerged for different reasons, around different problems and with different people has left me with different ways of looking at things and, when I put them together to prepare this talk, something seemed quite evident, at least to me: data and AI were now running through all those roads at the same time (I know, I have become a textbook nihil novum-ist).

The image that eventually helped me tell that part of the story was a flood (you see, as usual, looking for images helps me organise my thinking… and this time I wanted photographs, images of nature). But to get there I had to go much further back, to Web 2.0 and the enthusiasm with which many of us welcomed a web where suddenly we could not only find information, but also create, publish, share and connect with other people. What is much easier to see now is that, while we were creating, sharing and connecting, we were also producing data —huge amounts of data— so the same infrastructures that expanded our possibilities were also creating other conditions to which we were probably paying much less attention. In that sense, Web 2.0 was also a kind of Trojan horse for datafication, although saying that does not mean that all those possibilities for participation were somehow false, but precisely that both things happened together.

And from there I arrived at shaping, enacting, deciding, although a good part of the work involved in preparing the seminar consisted precisely in trying to decide what the hell those three verbs actually were. For a while, I tried to turn them into something much more orderly than they eventually became: three levels, three moments, three processes… perhaps a way of explaining how we move from conditions to action and from action to decision… as always, not linear or neatly lined up, but three verbs that allowed me, for a while, to separate things that in education inevitably appear entangled.

Shaping allowed me to ask how some possibilities become more available than others, and not only in terms of what we can technically do, but also what becomes easier to see, organise, justify, fund or even imagine. Enacting took me to what people actually do within those possibilities, and also upon them, because we can design conditions but then people interpret, use, modify, resist and find things to do with them that were not necessarily anticipated. And deciding, which was probably the verb I struggled with the most, had to be much broader than assessment or even teachers’ professional judgement, because I was also interested in all those decisions through which we determine what deserves our attention, what we consider relevant, what we want to make possible, what we protect or reject; ultimately, how something comes to matter. Writing them here, I can now see an analogy with Biesta’s three purposes of education (qualification, socialisation and subjectification…?) —that is not where they came from, and I don’t think they are exactly the same thing, but there you go, something else to keep thinking about.

And that was where preparing the talk became really complicated, because every time I found an explanation that seemed to put everything in order, ten minutes later it started bothering me. For a few days I thought that perhaps the problem I was trying to formulate was individualisation. After all, with AI we can increasingly imagine an educational experience built for one person: explanations for you, feedback for you, examples for you, an interlocutor permanently available to you, and there was something about that image that worried me. But turning individualisation into THE problem simplified things too much. Then I thought that perhaps what I needed to do was move away from the idea of student-centred learning and put relationships at the centre, but that didn’t convince me either, because the student remains absolutely fundamental and replacing one centre with another seemed to solve with a label something that was much more complicated. And at other points everything seemed to lead to teacher judgement, which made sense given my previous work and because educational judgement was even in the title, but that made shaping and enacting disappear far too quickly and, after preparing The Telescope, there was also the risk that deciding would end up becoming yet another story about assessment.

I think a good part of my nerves before the seminar came precisely from that. I didn’t have a thesis that I had decided to demonstrate and around which I could neatly organise the slides; I had several things that worried me and I was trying to find a way of putting them together without resolving them too quickly. In the end, rather than trying to close each of the three verbs with a definition, I ended up asking them questions. Three, specifically, which I also didn’t want to place neatly underneath each word, as if each question belonged to one of the processes, but rather to have them run through all three:

How do some possibilities become more available than others?

How do people act within and upon those possibilities?

How does something come to matter?

The first allowed me to think not only about who designs a possibility, but about how some things end up being easier to do, see, organise, justify or fund than others. The second kept agency at the centre, but an agency that takes place within conditions we have not entirely chosen and upon which, at the same time, we can act: what we do with a possibility can also change it. And the third was deliberately much broader than asking who makes a decision, because I was interested in how certain things come to command our attention, acquire value and become something worth pursuing, protecting or changing. And, of course, as soon as I started trying to answer any of the three, the entanglement appeared again: what we make available influences what we do, but what we do also changes the possibilities; what we decide matters modifies the conditions we create and the things we pay attention to; and what happens within those conditions can eventually change what we thought mattered in the first place.

But there was something else I didn’t want to disappear when talking about processes, conditions, technologies, institutions and agencies becoming entangled, and that was precisely the people who are inside all of this. Because shaping, enacting and deciding do not happen in the abstract. There are people inside them, with their histories, their experiences, their interests and their ways of understanding what is happening; people who agree and disagree, who interpret things differently, who change their minds and change other people’s minds, and who sometimes manage to make something together that none of them would have imagined on their own. And institutions are there too, not as the container in which education happens, but participating in what becomes possible, in what happens and in what eventually comes to matter.

I think it was precisely when I tried to put people back inside the picture that something else started worrying me. Because it is not enough for them simply to be there, or for us to have information about them. If participation means anything, it must also mean having some possibility of intervening in those processes: of modifying the conditions rather than simply acting within them, of changing what is happening rather than simply adapting to it, and of being able to contest even what others had already decided mattered. Obviously, we do not all have the same capacity to do this, nor the same responsibility —a student, a teacher, an institution, a government or a technology company occupy very different positions— but precisely for that reason, the question of who gets to intervene in shaping, enacting and deciding became increasingly important to me.

And it was from there, rather than from any generic defence of “the human” against AI, that the question of representations appeared. Data and AI allow us to build increasingly sophisticated representations of students, teachers, groups, institutions and learning processes. We can collect more traces, combine them, summarise them, interpret them and use them to act, and all of that can be extraordinarily useful. The question I started asking myself was what happens when those representations become so good and so useful that they begin to feel sufficient.

I can have an extraordinarily rich representation of a student and still need that student to be able to tell me no, that is not what happened to me, that is not what I was trying to do, or you have misunderstood what mattered to me. And an institution can have spectacular amounts of information about its teachers, their activity, their evaluations, their competences and their outcomes, and still need those teachers to participate in deciding what teaching should become.

That is where one of the few sentences that actually stayed with me almost as a statement in the seminar came from: Being represented is not the same as participating. And I think I now understand a little better why that sentence mattered to me. Participating is not simply being present in the data through which others shape conditions, interpret what is happening or decide what matters. Participating means being able to enter those processes and alter them, to say that the representation is not enough or that it is wrong, to introduce something that was not anticipated and, ultimately, to have some possibility of changing what becomes possible, what happens and what comes to matter.

I don’t yet know where that takes me, and I probably don’t want to know too quickly, because one of the good things that happened at Deakin was that the conversations afterwards moved some of the pieces again. The conversations with Luci Pangrazio and Chris Zomer about data literacy and how to think about teachers’ responsible use of data, the very long conversation with Margaret Bearman about assessment, teacher autonomy, professional judgement and what we need to keep open to that judgement, as well as the questions that emerged during and after the seminar, did not exactly help me close the problem, but they did move some of the things I had placed inside it. And I think that, after all, has quite a lot to do with the idea of participation with which I ended the talk.

It is now official: I have talked far too much… hahaha… but I’m afraid that is part of who I am… what can you do?

I’ll leave you with some of the slides I used… in case they say something to you…