Publications

Peer-Reviewed Archival Publications

2024

1. ACC

   ROMA-iQSS: An Objective Alignment Approach via State-Based Value Learning and ROund-Robin Multi-Agent Scheduling

   Chi-Hui Lin, Joewie J. Koh, Alessandro Roncone, and Lijun Chen

   In Proceedings of the 2024 American Control Conference (ACC)

   Abs arXiv PDF

   Effective multi-agent collaboration is imperative for solving complex, distributed problems. In this context, two key challenges must be addressed: first, autonomously identifying optimal objectives for collective outcomes; second, aligning these objectives among agents. Traditional frameworks, often reliant on centralized learning, struggle with scalability and efficiency in large multi-agent systems. To overcome these issues, we introduce a decentralized state-based value learning algorithm that enables agents to independently discover optimal states. Furthermore, we introduce a novel mechanism for multi-agent interaction, wherein less proficient agents follow and adopt policies from more experienced ones, thereby indirectly guiding their learning process. Our theoretical analysis shows that our approach leads decentralized agents to an optimal collective policy. Empirical experiments further demonstrate that our method outperforms existing decentralized state-based and action-based value learning strategies by effectively identifying and aligning optimal objectives.

2022

1. WS @ ACL

   Open-Domain Dialogue Generation: What We Can Do, Cannot Do, and Should Do Next

   Katharina Kann, Abteen Ebrahimi, Joewie J. Koh, Shiran Dudy, and Alessandro Roncone

   In Proceedings of the 4th Workshop on Natural Language Processing for Conversational AI (NLP4ConvAI @ ACL)

   Abs ACL PDF

   Human–computer conversation has long been an interest of artificial intelligence and natural language processing research. Recent years have seen a dramatic improvement in quality for both task-oriented and open-domain dialogue systems, and an increasing amount of research in the area. The goal of this work is threefold: (1) to provide an overview of recent advances in the field of open-domain dialogue, (2) to summarize issues related to ethics, bias, and fairness that the field has identified as well as typical errors of dialogue systems, and (3) to outline important future challenges. We hope that this work will be of interest to both new and experienced researchers in the area.

2020

1. IROS

   Cooperative Control of Mobile Robots with Stackelberg Learning

   Joewie J. Koh*, Guohui Ding*, Christoffer Heckman, Lijun Chen, and Alessandro Roncone

   In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

   Abs arXiv PDF

   Multi-robot cooperation requires agents to make decisions that are consistent with the shared goal without disregarding action-specific preferences that might arise from asymmetry in capabilities and individual objectives. To accomplish this goal, we propose a method named SLiCC: Stackelberg Learning in Cooperative Control. SLiCC models the problem as a partially observable stochastic game composed of Stackelberg bimatrix games, and uses deep reinforcement learning to obtain the payoff matrices associated with these games. Appropriate cooperative actions are then selected with the derived Stackelberg equilibria. Using a bi-robot cooperative object transportation problem, we validate the performance of SLiCC against centralized multi-agent Q-learning and demonstrate that SLiCC achieves better combined utility.

2. AAMAS

   Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation

   Guohui Ding*, Joewie J. Koh*, Kelly Merckaert, Bram Vanderborght, Marco M. Nicotra, Christoffer Heckman, Alessandro Roncone, and Lijun Chen

   In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)

   Abs arXiv PDF

   We consider solving a cooperative multi-robot object manipulation task using reinforcement learning (RL). We propose two distributed multi-agent RL approaches: distributed approximate RL (DA-RL), where each agent applies Q-learning with individual reward functions; and game-theoretic RL (GT-RL), where the agents update their Q-values based on the Nash equilibrium of a bimatrix Q-value game. We validate the proposed approaches in the setting of cooperative object manipulation with two simulated robot arms. Although we focus on a small system of two agents in this paper, both DA-RL and GT-RL apply to general multi-agent systems, and are expected to scale well to large systems.

2018

1. IEEE BigData

   Inline Detection of Domain Generation Algorithms with Context-Sensitive Word Embeddings

   Joewie J. Koh and Barton Rhodes

   In Proceedings of the 2018 IEEE International Conference on Big Data (IEEE BigData)

   Abs arXiv PDF

   Domain generation algorithms (DGAs) are frequently employed by malware to generate domains used for connecting to command-and-control (C2) servers. Recent work in DGA detection leveraged deep learning architectures like convolutional neural networks (CNNs) and character-level long short-term memory networks (LSTMs) to classify domains. However, these classifiers perform poorly with wordlist-based DGA families, which generate domains by pseudorandomly concatenating dictionary words. We propose a novel approach that combines context-sensitive word embeddings with a simple fully-connected classifier to perform classification of domains based on word-level information. The word embeddings were pre-trained on a large unrelated corpus and left frozen during the training on domain data. The resulting small number of trainable parameters enabled extremely short training durations, while the transfer of language knowledge stored in the representations allowed for high-performing models with small training datasets. We show that this architecture reliably outperformed existing techniques on wordlist-based DGA families with just 30 DGA training examples and achieved state-of-the-art performance with around 100 DGA training examples, all while requiring an order of magnitude less time to train compared to current techniques. Of special note is the technique’s performance on the matsnu DGA: the classifier attained a 89.5% detection rate with a 1:1,000 false positive rate (FPR) after training on only 30 examples of the DGA domains, and a 91.2% detection rate with a 1:10,000 FPR after 90 examples. Considering that some of these DGAs have wordlists of several hundred words, our results demonstrate that this technique does not rely on the classifier learning the DGA wordlists. Instead, the classifier is able to learn the semantic signatures of the wordlist-based DGA families.

Miscellaneous Publications

2023

1. XRDS

   Making Speech Recognition Work for Children: An Interview with Amelia Kelly

   Amy Adair and Joewie J. Koh

   XRDS: Crossroads, The ACM Magazine for Students

   Abs ACM PDF

   What does it take to build new AI technologies for education? Dr. Amelia Kelly, chief technology officer at SoapBox Labs, shares her experience with us in this interview.

2017

1. BA Thesis

   Algorithms for the Computation of Delaunay Triangulations

   Joewie J. Koh

   Reed College

   Link