The ongoing boom in tracking technology has enabled innovation in the security realm, such as facial recognition systems used by organizations like the TSA. Lesser known, however, is how the technology is used to track toddlers. 

Toddlers’ unpredictable movements and smaller number of defining features compared with adults have made them difficult to track. But a team of Northeastern researchers has recently taken steps to overcome these challenges, using multicamera tracking and new machine learning algorithms to better recognize and differentiate toddlers. 

While tracking toddlers might sound a bit ominous, the technology is intended to recognize features associated with neurological disorders that would be difficult for people to identify on their own. Early detection of these disorders can allow for more timely intervention, which can greatly reduce how much a given disorder might impact someone’s life as they get older. 

The project is a collaboration between Matthew Goodwin, a professor jointly appointed in Khoury College and the Bouvé College of Health Sciences, and Sarah Ostadabbas, a professor in the College of Engineering (CoE). The pair met through Northeastern’s Institute for Experiential AI (EAI), where they resolved to hire a postdoctoral researcher to assist them with automated detection of human behavior, especially as it relates to motor development and behavior of autistic individuals. 

That postdoc was Somaieh Amraee, who studies at CoE and is also a part of the EAI. She collaborated with Ostadabbas at the Augmented Cognition Lab — which Ostadabbas directs — and had a multiperson and multicamera tracking background that made her perfect for the project.  

Along with Amraee, the team recruited Bishoy Galoaa, an electrical and computer engineering master’s student who is being mentored by Amraee.  

“He was instrumental in creating the multicamera multiperson tracking algorithms in my lab,” Ostadabbas said. 

The team believes that tracking technologies could help detect early signs of autism spectrum disorder (ASD) in toddlers, which would be incredibly beneficial for parents and other caretakers of young children.  

“If you have an eight-hour video [of toddlers], imagine how time-consuming it would be to watch the entire footage,” Ostadabbas said. “Our goal is to develop computer vision technology that can automatically summarize such videos, extracting the most relevant moments. Additionally, we aim to create systems capable of real-time alerts, notifying parents or daycare personnel if specific critical actions or behaviors occur, ensuring a safer and more efficient childcare environment.” 

Many indicators of neurological disorders in young children can be seen in their movements. 

“In many pediatric disorders, some of the earliest features that cause concern about failing to meet development milestones … are impairments or differences in motor movement,” Goodwin said. “How are they proceeding from laying down to sitting up to crawling to walking to reaching to grasp? These are all motor signs with typical trajectories in the normative population … we can compare to.” 

Beyond not meeting development milestones, tics and other repetitive behaviors can also serve as early signs of potentially serious and even life-threatening disorders. Detecting such behaviors early is essential to providing families with the best support to help raise their children, according to Goodwin.  

“The brain is always rewiring, but we see the most change in neurological development in the first years of life,” said Goodwin. “So, if children are missing developmental milestones or are on a different trajectory early on, the sooner we can identify them and provide intervention that promotes developmental outcomes, the better. The later we wait to intervene, the less neural rewiring based on environmental support we see.” 

The team believes that more automated methods of tracking toddler behavior may allow for more equitable diagnosis of these disorders, many of which, including ASD, have sharp race-and gender-based inequalities when it comes to diagnoses. 

“The American Academy of Pediatrics has mandatory screening for autism at 12, 24, and 36 months, even though we don’t diagnose officially until 36 months,” Goodwin said. “The average age of diagnosis is usually three to four years old, but if you’re not white, it’s eight years old. These kids are not coming to clinics in an equitable way.” 

The team began developing their toddler tracking technology by addressing the major roadblocks that previous researchers faced. First was a lack of publicly accessible video data on toddlers due to privacy and security issues. Ostadabbas had encountered this challenge on a previous project involving infants and computer vision, an endeavor that eventually won her the National Science Foundation CAREER award with Goodwin as a mentor. 

“We learned from that study you could [pull extra video data] from social media and use it to test your model,” Ostadabbas said.  

The team also added data by working with the Marcus Autism Center, a Georgia-based clinic and close collaborator of Goodwin’s that specializes in intervention and evaluation for children with ASD. 

After getting the data they needed, the team had to address another serious challenge: developing successful multicamera tracking. This method would allow for a more comprehensive view of each toddler, but with the increased number of cameras came increased potential for complications. Many multicamera tracking technologies work by “using the average” of the information they receive from each camera. While this sounds practical in theory, in reality these averages are often muddied by major outliers such as an item blocking one camera’s view.  

“We realized that adding an extra camera doesn’t just provide more information. It also introduces more challenges that we have to be mindful of,” Ostadabbas said. “It requires using them intelligently to make sure the result is robust and less error is introduced to the tracking problem.” 

To address the issue, the researchers created an algorithm that presented a more representative view of the data. They developed a genetic algorithm that mimics the process of natural selection to continuously optimize the outputs of an algorithm until more and more accurate results are reached. And they added two data modules, one focused on movement and another on feature recognition, to help the algorithm differentiate toddlers and avoid confusion based on their unpredictable movements.  

Their success represents a step forward in toddler tracking technology, particularly within preventive health care. Even beyond the tracking technology itself, much of the data used for the project will likely have lasting impacts on the field. 

“[The team] is creating data sets that families have given them permission to share with other investigators,” Goodwin said. “That’s going to help other people who are developing tracking methods; they can work with the same data and compare and contrast their performance with the performance presented in our paper.” 

Olivia Mintz contributed reporting to this story. 

Khoury College doctoral student Anthony Sicilia knows firsthand how quickly large language models (LLMs) and conversational AI are advancing. While he sees this as positive, he also believes it’s important that AI safety protocols progress at the same rate.  

Alongside his advisor, assistant professor Malihe Alikhani, Sicilia focuses on addressing uncertainty and eliminating bias in conversational AI. Similarly, Alikhani’s work emphasizes increasing accessibility and inclusivity in AI language models by integrating social science and cognitive science with machine learning.  

In November, the pair, along with their coauthor Sabit Hassan, received two outstanding paper awards from workshops at the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) in Miami. The papers focus on how LLMs can lead to safer user experiences when deployed in online web forums and other real-world environments.  

“AI safety needs to be a central focus of research, especially as conversational AI is getting so good and more and more people are interacting with it,” Sicilia said. “A specific focus of AI safety in these papers is how user behaviors can be influenced by large language models.” 

For the first paper, “Eliciting Uncertainty in Chain-of-Thought to Mitigate Bias against Forecasting Harmful User Behaviors,” the researchers used Reddit conversations to test the accuracy of five LLMs in predicting personal attacks and other harmful user behavior on social media. They found that while the models tended to have a bias against predicting harmful events, this bias could be reduced by asking the model to rate the likelihood of a personal attack on a 10-point scale based on a conversation fragment, and by asking it to explain its reasoning in a chain-of-thought prompt such as “let’s think step by step.”  

Sicilia and Alikhani believe that their findings could help make social media a safer place, ensuring that potentially hostile interactions don’t slip through the cracks of existing moderation systems. 

“The findings demonstrate how asking language models to represent their uncertainty can reduce biases and improve accuracy, especially when working with limited data,” Alikhani said. “This is particularly important for applications like social media moderation, where biases can have real-world consequences.”  

The second paper, “Active Learning for Robust and Representative LLM Generation in Safety-Critical Scenarios,” stemmed from a project Sicilia and Alikhani worked on together last year for Amazon’s Alexa Prize TaskBot Challenge, in which their team placed third. The competition required participants to create a task-oriented AI assistant or “TaskBot” that interacted with real Alexa users. To refine the TaskBot’s safety system, the team created an LLM-generated simulated data set with 5,400 potential safety violations using active learning and clustering. This process trained the model to anticipate safety concerns for users in a wide range of situations based on their dialogue.  

READ: Khoury inclusive AI team places third in Amazon’s Alexa Prize TaskBot Challenge 

“Whenever the safety system senses the user is talking about something that it should not respond to, or that is unsafe and might require a 911 call, then it would trigger some template response that says, ‘This is out of my jurisdiction. You need to seek out help,’ or something like that,” Sicilia explained. 

Potential safety violations ranged from health emergencies to complex legal problems to self-harm, none of which language models are equipped to handle. Despite the relative rareness of these situations, Sicilia and Alikhani believe that it’s important for AI training to factor in the possibility of unexpected safety-critical scenarios. What’s more, Sicilia noted that the data set the team created for the paper could be useful for training other safety systems, so they’ve made it publicly available. 

The team’s hard work culminated in their presenting of their papers at EMNLP, where both earned outstanding paper awards.  

“I wasn’t necessarily surprised, I thought they were pretty strong works,” Sicilia said of the win. “But it’s nice to receive the recognition, and I think it’s important work.”  

Sicilia is in his final year as a doctoral student and hopes to start his own lab, which will continue research to improve the communication capabilities of conversational AI models. Likewise, Alikhani was overjoyed at the success of both papers and wants to continue pursuing projects that advance AI accessibility, in line with Khoury College’s core mission of “computer science for everyone.” 

“AI should benefit everyone, not just a select few,” she said. “This means creating technologies that are inclusive and representative, with a particular emphasis on safety-critical applications.”

This fall, Khoury College will introduce a redesigned sequence of introductory computing courses. 

The new three-course sequence preserves the design-first approach to programming — including the “design recipe” — that has proven valuable across several generations of students. It also retains the rigor, in-person labs, and pair programming of the existing courses. But it also adds flexible curriculum pathways for programming beginners and veterans alike, balances the workload more evenly across the three courses, incorporates ethical considerations and discussions throughout the curriculum, and updates course content to reflect advances in computing education, programming tools, and employer needs. 

“Our goal is to provide a rigorous introductory computing education and prepare students for both industry careers and graduate study,” Dean Elizabeth Mynatt says. “With these changes, we retain the core ideas that have made our program successful while leveling up the curriculum to reflect advances in the field.” 

Course designers have incorporated feedback from college faculty, leaders in the Center for Inclusive Computing, and Khoury student advisory boards. Going forward, designers will continuously evaluate and revise the courses in response to student outcomes, community feedback, and advances in computing education. 

“This is the first major shift of the intro sequence in a long time,” explains Christo Wilson, associate dean of undergraduate programs. “After a thorough and deliberate curriculum revision, we are confident that the courses retain their rigor, ready students for co-op work, provide a solid foundation for advanced classes, and position all students for success right from the start.” 

To learn more about the upcoming changes, click on one of the links below, or simply read on. 

• Placement process 
• Introduction to Program Design and Implementation (CS 2000) 
• Program Design and Implementation 1 (CS 2100) 
• Program Design and Implementation 2 (CS 3100) 
• Frequently asked questions 

Placement process 

What’s changing: Starting in May 2025, all Khoury students — including first-years and transfer students — can take an online self-assessment to determine if they should place out of the introductory course, CS 2000. Students who pass will begin with “Program Design and Implementation 1” (CS 2100), where they will need to clear in-class screenings early in the semester to confirm their placement. 

The self-assessment will cover programming topics that are exclusive to the introductory course, but not the topics that reappear in the next course as well. Multiple programming languages, including Python and Java, would suffice for use on the self-assessment. 

AP or IB credit will not exempt a student from the introductory course. However, the college expects that many students who have such credit will be able to pass the self-assessment. 

Why: Separating true beginners from those with programming experience will provide a more supportive environment for the former and an accelerated pathway for the latter. 

Thanks to the proliferation of high-school computing courses, more students are entering Khoury College with computing experience. By skipping to the second course, these students can challenge themselves in a class where the material and pace are better suited to their experience level. In doing so, they also earn back a course slot, freeing up an extra slot for a higher-level elective. 

At the same time, access to computing in high schools is not evenly distributed, with students in affluent areas more likely to get those opportunities. In keeping with Khoury College’s mission of “CS for everyone,” the first course will be reserved for those who did not or could not study computing before college, ensuring that they feel welcome and that instructors can engage the entire room at the same level. 

Introduction to Program Design and Implementation (CS 2000) 

Replacing: “Fundamentals of Computer Science 1” 

Course Description: Introduces computer science and data science to students with no programming experience. Starts by building programs with numbers, text, and images, then moves to exploring real, complex data sets both interactively and through coding. Students then practice coding using a popular industrial language with a professional programmer’s interface to the code. Students learn to identify and respond to ethical challenges in program design. 

What’s staying the same: The course will use a teaching language to introduce students to the systematic design of programs, then build gradually to a full-featured language. Students will cover structural recursion and receive support for images, testing, and carefully crafted error messages design. 

What’s changing: For the first 8–9 weeks, the course will be taught in Pyret, a teaching language built by CS educators who understand the successes and limitations of Racket teaching languages. In addition to first-class support for tables and data science, better support for testing, and a more modern environment with better errors, Pyret facilitates a smooth transition to Python for the last 4–5 weeks, once students have learned foundational computer and data science concepts. In addition to explicit coverage of ethics and new exposure to data science — including operations, transformations, and visualizations of tabular data — students will learn about mutation and iteration. Some of the more advanced topics, including generative recursion, will be deferred to the second and third courses. 

Program Design and Implementation 1 (CS 2100) 

Replacing: “Fundamentals of Computer Science 2” 

Course Description: Building on intro programming experience (from CS 2000 or a different institution), examines the fundamentals of program design and implementation. Studies design of data and object-oriented programs, including common patterns, use of data structures, and underlying principles such as abstraction, encapsulation, inheritance, and interfaces. Introduces common software engineering practices such as test-driven development, version control, development environments, and good programming habits. Practices using these design principles by writing medium-sized applications and using data science code libraries. Continues to interweave ethical challenges and skills in program design. 

What’s staying the same: In the context of a real-world language, students will systematically design programs while incorporating existing libraries into their design. Students will also be introduced to object-oriented programming. 

What’s changing: As students can now place out of the introductory course, this course will cover all of the systematic design concepts that it introduces, rather than assuming that students are already familiar with them. This course will be taught in Python, allowing students to learn core data science design patterns and libraries. The course will also expand on version control and development environments, preparing students to work on increasingly large programs. And by taking on material that previously populated the third course, this course helps to enable a more even balance of concepts and workloads across the courses in the sequence. 

Program Design and Implementation 2 (CS 3100) 

Replacing: “Object-Oriented Design” 

Course Description: Building on CS 2100 foundations, examines program design at increasing scales of complexity. Reviews abstraction, encapsulation, inheritance, and interfaces in statically typed object-oriented languages. Presents a comparative approach to software design patterns and paradigms, including object-oriented and functional programming. Fosters a deeper understanding of program design principles, including interface design, test-driven development, graphical design notations, reusable software components, and open-source ecosystems. Illustrates the impact of design-time decisions on software correctness, including accessibility, changeability, performance, reusability and privacy. Students collaborate to design and implement a large software project. 

What’s staying the same: The course will center on object-oriented design patterns and bigger-scale programs in Java. Students will program in pairs and learn the importance of writing maintainable programs by reviewing other students’ code. 

What’s changing: The course will start with a rapid introduction to Java — as incoming students will no longer be expected to know it — and will provide a comparative discussion of program design in Python and Java. Given the challenging nature of the main project, new lectures and labs will fill in gaps and help students build larger applications. The course will also cover modern topics like open-source ecosystems and composing new programs from existing libraries. Thanks to new intro material on memory management and user-centered design, students will be better prepared for higher-level courses in systems and human–computer interaction. Students will analyze trade-offs between nonfunctional requirements such as reusability, changeability, accessibility, performance, and privacy. Software testing concepts will be embedded throughout as students consider the ways that their software might fail. 

Frequently asked questions about the new curriculum 

What changes are still being determined and how can I offer feedback? 

Course titles, languages, and high-level concepts have been cemented based on feedback from students and faculty. As course designers work to solidify curriculum specifics, community members who want to weigh in on these changes before the fall can do so in a few ways: 

• Email Christo Wilson 
• Attend town halls (all times EST) 
  • Faculty: February 6 at 12 p.m. 
  • Students and TAs: February 7 at 1 p.m. (International Village 019 + Zoom) and February 21 at 4 p.m. (Zoom). Announcements with Zoom links will be sent to undergraduates. 
• Participate in a Reddit AMA on February 4 from 1 to 3 p.m. with course designers Jonathan Bell, Rasika Bhalerao, and Daniel Patterson. 
• Work with the college for summer co-op, helping to build and test new labs, assignments, and infrastructure. Interested students can contact Wilson. 
• Participate in working groups to refine key policies (faculty only) 
• Share with our co-op faculty how our learning outcomes relate to on-the-job success (employers only) 

How will the redesigned intro sequence prepare students for co-op? 

While keeping the tenets that have served students and co-op employers well, the curriculum will now expose students to Python and Java — the two programming languages most requested by employers — in their first few semesters. Students will also benefit from explicit training in software development tooling and deployment environments, which are requested by employers. 

Will computer science, data science, and cybersecurity majors be impacted differently? 

CS, DS, and cybersecurity students will take — or place out of — the same intro course (CS 2000), reflecting the principle that the first step in designing a program is understanding the data behind it. Students will then split off into separate tracks: CS and cybersecurity students to the CS intro sequence, DS students to the DS sequence. Switching between the CS and DS sequences will be simpler since the second classes are both taught in Python and cover similar topics. 

Khoury faculty will continue to examine whether additional CS and DS courses can be merged in the future. 

What will happen to students who are partway through the current sequence?

“Fundamentals of Computer Science 2” will be offered in summer 1, while “Object-Oriented Design” will be offered in summer 1 and fall.

How will things change for transfer students?

Previously, transfer credits often counted for credit hours but did not fulfill computer science major requirements, meaning transfer students needed to restart their major studies. Now these students can place out of the intro class through the self-assessment. 

---