Building Child Safety into Digital Platforms
Jennie Ito, a developmental psychologist and expert in children’s media and content appropriateness, joins Laura and myself in a compelling conversation on creating safer digital environments for kids. Drawing from her vast experience, which includes spearheading content policy development for kids’ experiences on YouTube, Google Play, and her current role at Roblox, Jennie provides a deep dive into the intricacies of age-appropriate digital policy-making. Together, they explore the critical challenges and innovative solutions in ensuring child safety in digital platforms.
Jennie: My name is Jennie Ito and I’m a senior product policy manager at Roblox. Before that, I was at YouTube and Google where I was a policy specialist on the Google Play Store. I was also a user experience researcher for a short period of time on the Google Kids and Family team and then I went back to working on policy at YouTube, where I led policy development for kids, tweens and teens for YouTube and YouTube Kids.
Jordan: So you’re at Roblox, which means the two of you work together. You know each other.
Jennie: Yes, that’s right. We do work together.
Jordan: So do you spend lots of time working together?
Laura: We do. In fact, I’m very, very lucky. So, Jennie, she’s going to talk a little bit about her work that she does at Roblox, which is amazing. But the work that I do at Roblox is all around civility and safety education, and it’s amazing how many times Jennie’s name comes up in a conversation where I’m like, you know who we need to talk to?
Laura: Jennie, and so we’re really pleased that she’s been able to support us with lots of different projects that we’re working on, some of them that are, you know, cross-functional working with lots of different academics, experts in the field, other tech companies. And then some that are much more internal projects. But yeah, I pretty much say there’s not a day goes by when we don’t email or chat to each other, so it’s such a pleasure to have you here, Jennie.
Jordan: You’re gonna have to explain to me, what do you actually do that makes your name come up so often in Laura’s conversations?
Jennie: Well, as Laura mentioned, she’s really focused on civility, and my team is really focused on safety. So we like to see we kind of write the rules of Roblox. And so we’re really focused on keeping. And all our users actually because our platform is not just focused on kids’ safety. And so there’s a lot of obvious overlap between civility and safety. So, as Laura mentioned, we work quite closely together.
Jordan: And so what does your day look like when you’re working on safety like? What does that mean for people who don’t know anything about what it looks like behind other than the app on their phone, or their computer, or their tablet, or their console? What does it mean for someone to work on safety?
Jennie: Yeah, that’s a great question. So I would say no two days of my days really look alike. There’s lots of lots of different things that I work on, which makes my job really fun and exciting. So one of the first things when it comes to safety is our team drafts, policies and then, which I was mentioning are kind of like the rules of Roblox, and then we work really closely with the moderation team which actually helps enforce those rules and make sure that everyone is is following the rules on the platform. So a big part of my job is doing research and coming up with these policies and then working with our cross-functional stakeholders to you know whether it’s product or legal to to actually make sure that these these policies make sense, and that they truly are going to keep our users safe on the platform.
Jordan: But you come to this as a child psychologist, right?
Jennie: Yes, that’s right. Yes, my background’s in developmental psychology.
Jordan: So I’m curious. Can you tell me a bit about how that impacts all these everyday decisions? How, does, how like, I just don’t think that most people are imagining that there’s like child psychologists sitting there in meetings thinking about thinking, and so and I know there are. And Lauren knows there are so like. What is that like? What, what your parents? How would you explain to parents? How like, how like how you get involved in that, and how you bring real developmental research to these questions about your digital well-being?
Jennie: Yeah, that’s a good question. So I will say that this is something that definitely is more common now. But when I started in tech back in 2015, it actually wasn’t that common to have child development, or even educational specialists in the room, or have a seat at the table. Thankfully, this has changed over the years. So both at my job at YouTube, but also at Roblox. As I mentioned, our users are not just adults. We also have a large proportion of our users are actually children and tweens and teens. And when it comes to working or working with policies around kids and tweens and teens, you not only have to think about kids. You also have to think about parents, which is, you know, an additional challenge. So it’s definitely something I’m thinking of when we talk about users when it comes to kids policies. We’re thinking about kids as well as parents and it’s really important when we’re developing policies, you know, children are not just mini-adults, so we have to take what developmental stages that they’re at into consideration.
So that’s always kind of where I start. My policy development is really focusing on different developments. You know what changes are happening in children’s social development, cognitive development, how that actually can shape their media preferences, and also what type of content might be more risky for them, depending on their developmental stages. So really thinking through all of this, as we’re as we’re writing our policies.
Jordan: Yeah, that’s fascinating.
Laura: It is absolutely so. One of the things I’d really love to understand. There’s kind of two parts to this. So one is, how do you, tailor those community standards, the rules for a platform, so that they are understandable and digestible by both the parents and adults in their lives, and also the kids and teens, and the second part being, how do you you know, you kind of mentioned developmental milestones and things like that. How do you keep your finger on the pulse of what’s happening out there that’s going to affect these kids and teams to build those community standards?
Jennie: Yes, it’s definitely challenging. Well, I had two tweens myself, actually I’d rather say one tween and a teenager, because he’s now 13. So I definitely am reminded every day what’s cool and what’s not cool by my children, and also I’m in the club. Yeah. So I have that. But to answer your second question first, definitely working with different academics, staying up to date on developmental research. This is actually one of my favorite parts of my job when I do get to interact with other experts.
I work with different academics, staying up to date on developmental research. This is actually one of my favorite parts of my job when I do get to interact with other experts. And when it comes to the community standards, it’s definitely a challenge, because, especially at a platform with Roblox and YouTube as well where we had users of all different ages. We have to make sure that our community standards are understandable to our youngest users. They’re written in a way that works with all the different age ranges which can be definitely difficult.
Jordan: Yeah. So I’m curious about how you think about this. You know, with the younger kids, of course, younger kids follow rules pretty well. They like to understand them. But as you start to get older, as you get into the tweens and teens, especially at a place like any digital platform where it feels like their playground, they just see rules as obstacles. They’re all mad about them. They don’t understand that any of them are for them, or they’re just in the way of fun. Right?
So, what do you think about that? As you’re starting to think about policy? Because, obviously, we want people to understand it. We want them not to feel it as something that’s like a reason to be angry at my space. My playground.
Jennie: Absolutely. So there’s a couple of things one is for really starting to think about how we get feedback. If someone does violate one of our policies. How can we tailor that feedback to different ages? And how can we focus more on education and deterrence versus kind of like a punishment, because we know, especially with tweens and teens, they’re going to break the rules. They’re going to push the boundaries, you know. So how can we communicate with them? You know this actually isn’t allowed, but we’re going to give you another chance to correct that behavior. And another thing is really working very closely with the user experience research team there, doing research with parents and kids and twins and teams, and really listening to what they want. You know what they think the rules should be, and just like incorporating a lot of that feedback where we could, and not just after the fact of getting feedback after we’ve made up the rules, but thinking about taking a child-centric approach from the beginning and listening to what they want.
Laura: As a parent of a teenager as well, there’s a common theme here, isn’t there, when we hear that we know it’s perfectly normal for tweens and teens to push the boundaries to go seek out inappropriate content and sort of to experiment when they’re online. I know that that’s kind of terrifying to a lot of parents who feel really out of their depth. Is there any advice that you could give to just help parents feel empowered in how to manage that situation?
Jennie: Yeah, I definitely can relate to how scary that can be. I think, having open conversations with your kids about your twins and teens about what they could see online, and then also what they should do if they do see something that makes them feel uncomfortable. You know, if they engage with someone, and it makes them feel comfortable to report that behavior. If they see something, they’re not really sure how to make sense of it, talk to your parents. And also not to overreact as parents when our kids do stumble upon something that they probably shouldn’t be seeing. Because, as you mentioned, it is very developmentally typical for children to seek out this type of content. But it also can be, you know, even Sonya Livingstone’s work about what bothers teens and tweens online. You know a lot of kids stumble upon things they didn’t even know existed like, maybe hyper-sexualized content, or something that’s really scary in the news. And so I think it’s really important to have those conversations with your kids.
Jordan: So when you look at the entire landscape of the platforms that kids and teenagers are using, you know, I don’t want to call any specific one out, because I’m gonna ask you a question here. But any of them, whether we’re talking about Roblox, or YouTube or Facebook or Instagram. Where do you think are the places where we really need to think about? You know, I mean as adults and kids. Not as companies, but as adults raising kids.
Jennie: Yeah, I think when we look across all platforms, I would say, probably in the tween, and in teen years, especially when you hit 13. I know for myself. As I mentioned, I have a 13-year-old. I had, you know, parental controls on a lot of these different platforms, and then suddenly, at 13 they disappeared completely. And so that’s a real shift. And so I think, either having. You know whether it’s, you know, just really having that communication with your child, but also giving parents the tools to have those conversations. You know some of these platforms. Maybe we need some more extended defaults and controls into the, you know, early teen years because I do think it’s quite a shift where you know there’s all these with really restrictive controls, and then all of a sudden at 13 they’re all completely gone, and if a child has been very restricted, it can be very overwhelming to kind of have everything open up to them with no limits.
Jordan: Yeah, I used to say all the time that everybody gives the kids the phones at 13, and I’m like, you should start with, maybe some boundaries, younger right?
Jennie: Yeah, definitely. That was definitely something that you know — and it’s funny, it hadn’t really occurred to me till I got that email, saying, hey, by the way, parental controls are no longer on these accounts.
Laura: Amazing. And I’d love to say thank you so much for mentioning one of our dear friends, Sonia Livingstone. She’s actually previously been on series 1 of our podcast. So you’re in great company and everyone else is great. And then, yeah, absolutely thanks. Thank you for the plug. This has been. This has been such a lovely conversation, and I’m sure you know a lot of our audience who our parents will find really reassuring that what they’re experiencing is just normal. That’s the first thing that makes everyone feel better. And you’ve definitely given some really fantastic advice as well. I’m gonna take us a little bit into the future now. So thinking about new innovations, actually one of the projects you and I are working on at the moment is about this kind of thing. What can we do in the next 5 to 10 years? So what innovations do you want to see? What are you? What new things do you feel should be coming, or might be coming, that is going to make a lot of this stuff easier for parents and kids to navigate.
Jennie: You know. I don’t know if this is a great answer, but I think one of the things I’ve been thinking a lot about, especially at Roblox and kind of this whole metaverse is thinking about. How can we have children and adults interacting in a safe way? I’m in the same space and a shared space. You know, because a lot of our other platforms, we kind of have our children siloed, separate from adults, but we are moving towards this world where they want everyone interacting together, and that just has so many safety challenges.
Laura: But I also think it could be, it can be like the opportunity. It is pretty amazing for shared experiences, but I do think it’s definitely a challenge. It is. But actually, online spaces are a reflection of the real world, and we don’t just keep kids locked up in a house. They are as our tweens and teens that we’ve already mentioned, they are going out. We send them to the store. They’re going to the park by themselves. They’re riding their bikes to school. They do have to interact with all sorts of strangers. So yeah, I think you know for me, certainly the responsibility on the platform to make sure that we can, that we’re thinking about that, and putting tools in place and helping to educate people around. You know, building, resilience and recognizing risk and harm, and all of those things. I think it’s really important.
Jennie: But yeah, it’s great to hear that you’re really thinking about this because you’re the person who needs to be thinking about that. And understanding kind of the, you know, unique risks that are online that are separate and different from the real world. And you know, the only the only, the only the best way, at least, that that kids are gonna learn. It is to be able to interact in these spaces with their parents. It’s where we are, it’s where we learn most of our most of what we learn about etiquette and so I wish there were more. I wish there were more spaces younger, and I wish there were more spaces older, older for it to happen. What do you think, Jennie? Like? What are the blind spots, or the places where the way that we think about the digital world, the way we think about the metaverse, where we think about online digital play. Where do you think we have blind spots that actually keep us from making the progress that we really need?
Jennie: I am happy to see kind of more developmental psychologists, people who really understand kids and tweets, really developing like the strategies behind these products and kind of really helping with product development. Because I think that’s really going to help us understand and kind of build these spaces with kids in mind. I make sure that they actually are appealing to them. And they want to be in these spaces, I think, for a long time it was. You know we didn’t have this expertise and we weren’t involving children from the beginning and listening to their feedback, and so I hope that we continue to do that as we build out these products.
Laura: Yeah. Same, I think that’s vital, isn’t it. And you know we know that being part of the product has been a thing with tech spaces and platforms for such a long time and we have to be even more thoughtful and careful about our duty of care. What? When it’s got young people involved as well. So okay, a bit more personal. So what’s next for you? What big dreams, what things are you working on, whether at Roblox or outside of that? What are you currently working on?
Jennie: Yeah. So I can. I mentioned something. I think I’m allowed to mention this. If not, we’ll have to edit it out, Laura. But right now I’m working on aging up the platform at Roblox, and one of our biggest growth areas is with our users from 17 to 24. So I’m really excited about that. I think in the past it’s been more focused on our younger users. But now we’re focusing on tweens and teens, and just making sure that we have appealing content for them, and making this obviously continue to make this a safe space for them. And so I did something similar over at YouTube, and I really enjoyed that project because I especially as a mom of a between and team. It’s exciting to kind of be building a space. I know my own kids are going to enjoy it.
