The presentation offers a comprehensive overview of the current landscape of Large Language Models (LLMs). It begins by examining recent advancements and trends in LLM research, highlighting key innovations that have shaped the field. The talk then shifts focus to practical applications, demonstrating how LLMs are being utilized across various industries to solve complex problems. The talk also includes a summary of safety and ethical considerations associated with LLM deployment, addressing concerns such as bias, privacy, and misuse. Finally, the talk concludes with forward-looking predictions, speculating on the future trajectory of LLMs and their potential impact on society.

Research studying the prediction of antibiotic resistance based on mass spectrometry data and machine learning focuses only on simple models for the identification of resistance to one antibiotic at a time, Even though a problem of multidrug resistance is currently being faced. Therefore, in this study, a multi-label approach for classifying multidrug resistance in Escherichia coli samples using raw MALDI-TOF mass spectrometry data and deep learning techniques was developed. The spectra from a recently published public database, encompassing over 4,500 samples of the bacteria under study, were utilized, sufficient for training a deep learning model, specifically a one dimensional convolutional neural network for this case. The use of this architecture proves to be highly efficient, achieving weighted AUROC and AUPRC values equal to or greater than 0.80, as well as a general performance calculated using the Hamming loss metric reaching 0.132. These results demonstrate that the use of deep learning allows for the development of complex models that enable the simultaneous identification of a predefined set of antibiotics, aiding in the determination of a highly effective treatment.

Accurate medical imaging segmentation is critical for precise and effective medical interventions. However, despite the success of convolutional neural networks (CNNs) in medical image segmentation, they still face challenges in handling fine-scale features and variations in image scales. These challenges are particularly evident in complex and challenging segmentation tasks, such as the BraTS multi-label brain tumor segmentation challenge. In this task, accurately segmenting the various tumor sub-components, which vary significantly in size and shape, remains a significant challenge, with even state-of-the-art methods producing substantial errors. Therefore, we propose two architectures, FMG-Net and W-Net, that incorporate the principles of geometric multigrid methods for solving linear systems of equations into CNNs to address these challenges. Our experiments on the BraTS 2020 dataset demonstrate that both FMG-Net and W-Net outperform the widely used U-Net architecture regarding tumor subcomponent segmentation accuracy and training efficiency. These findings highlight the potential of incorporating the principles of multigrid methods into CNNs to improve the accuracy and efficiency of medical imaging segmentation.

Drug repositioning, the identification of novel uses of existing therapies, has become an attractive strategy to accelerate drug development. Knowledge graphs (KGs) have emerged as a powerful representation of interconnected data within the biomedical domain. While link prediction on biomedical can ascertain new connections between drugs and diseases, most approaches only state whether two nodes are related. Yet, they fail to explain why two nodes are related. In this project, we introduce an implementation of the semi-parametric Case-Based Reasoning over subgraphs (CBR-SUBG), designed to derive a drug query’s underlying mechanisms by gathering graph patterns of similar nodes. We show that our adaptation outperforms existing KG link prediction models on a drug repositioning task. Furthermore, our findings demonstrate that CBR-SUBG strategy can provide interpretable biological paths as evidence supporting putative repositioning candidates, leading to more informed decisions.

Geological storage of CO$_2$ is a key climate change mitigation strategy. As important as location selection and injection planning is monitoring that the gas is contained for long periods of time. We introduce a fully 3D, deep learning-based approach for the joint inversion of time-lapse surface gravity and seismic data for reconstructing subsurface density and velocity models. The target application of this proposed inversion approach is the prediction of subsurface CO2 plumes as a complementary tool for monitoring CO2 sequestration deployments. Our joint inversion technique outperforms deep learning-based gravity-only and seismic-only inversion models, achieving improved density and velocity reconstruction, accurate segmentation, and higher R-squared coefficients. Future work will focus on validating our approach with larger datasets, simulations with other geological storage sites, and, ultimately, field data.

Epidemic forecasting is a crucial tool for public health decision making and planning. There is, however, a limited understanding of how epidemics spread, largely due to other complex dynamics, most notably social and pathogen dynamics. With the increasing availability of real-time multimodal data, a new opportunity has emerged for capturing previously unobservable facets of the spatiotemporal dynamics of epidemics. In this regard, my work brings a data-centric perspective to public health via methodological advances in AI at the intersection of time series analysis, spatiotemporal mining, scientific ML, and multi-agent systems. Toward realizing the potential of AI in public health, I addressed multiple challenges stemming from the domain such as data scarcity, distributional changes, and issues arising from real-time deployment to enable our support of CDC’s COVID-19 response. This talk will overview our developments to address these challenges with novel deep learning architectures for real-time response to disease outbreaks and new techniques for end-to-end learning with mechanistic epidemiological models—based on differential equations and agent-based models—that bridge ML advances and traditional domain knowledge to leverage individual merits.

This presentation will describe the challenges of landing on Mars and how those challenges were addressed in NASA missions through the years in response to ever-increasing science requirements, previous landing experiences, and changing programmatic constraints. From the Viking legged lenders, through the Mars Pathfinder and the Mars Explorations Rovers airbag landers, and ultimately the Curiosity and Perseverance SkyCrane landers, Mars exploration has provided fertile ground for the development of innovative landing technologies in our quest for extracting the scientific secrets hidden in the Red Planet.

The actual time is from 15:30 - 16:30, but the event scheduler is not allowing it. This is a joint poster session with the other affinity groups.

In the last few years, three major topics received increased interest: deep learning, NLP and conversational agents. Bringing these three topics together to create an amazing digital customer experience and indeed deploy in production and solve real-world problems is something innovative and disruptive. We introduce a new Portuguese financial domain language representation model called BERTaú. BERTaú is an uncased BERT-base trained from scratch with data from the Itaú virtual assistant chatbot solution. The novelty of this contribution lies in that BERTaú pretrained language model requires less data, reaches state-of-the-art performance in three NLP tasks, and generates a smaller and lighter model that makes the deployment feasible. We developed three tasks to validate our model: information retrieval with Frequently Asked Questions (FAQ) from Itaú bank, sentiment analysis from our virtual assistant data. All proposed tasks are real-world solutions in production on our environment and the usage of a specialist model proved to be effective when compared to Google BERT multilingual and the Facebook’s DPRQuestionEncoder, available at Hugging Face. BERTaú improves the performance in 22% of FAQ Retrieval MRR metric, 2.1% in Sentiment Analysis F1 score. It can also represent the same sequence in up to 66% fewer tokens when compared to "shelf models".

Deep neural networks are vulnerable to adversarial attacks, making adversarial robustness a pressing issue in deep learning. Recent research has demonstrated that even small perturbations to the input data can have a large impact on the model's output, making it susceptible to adversarial attacks. In this work, we introduce Delta Data Augmentation (DDA), a data augmentation method that enhances transfer adversarial robustness by sampling perturbations extracted from models that have been robustly trained against adversarial attacks. Our work gathers adversarial perturbations from higher-level tasks instead of directly targeting the model. By incorporating these perturbations into the training of subsequent tasks, our method endeavors to augment both the robustness and adversarial diversity inherent to the datasets. Through rigorous empirical analysis, we demonstrate the advantages of our data augmentation method over the current state-of-the-art in adversarial robustness, particularly when subjected to Projected Gradient Descent (PGD) with L2 and L-infinity attacks for CIFAR10, CIFAR100, SVHN, MNIST, and FashionMNIST datasets.

Seismic advances in generative AI algorithms have led to the temptation to use AI-synthesized data to train next-generation models. Repeating this process creates autophagous (“self-consuming”) loops whose properties are poorly understood. We conduct a thorough analysis using state-of-the-art generative image models of three autophagous loop families that differ in how they incorporate fixed or fresh real training data and whether previous generations’ samples have been biased to trade off data quality versus diversity. Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease. We term this condition Model Autophagy Disorder (MAD) and show that appreciable MADness arises in just a few generations.

AI-enabled precision medicine promises a transformational improvement in healthcare outcomes, however training on biomedical data presents a challenge: it is often high dimensional and clustered, with limited observations. To overcome this, we propose a simple and scalable approach for cluster-aware embedding that combines embedding methods with a convex clustering penalty. Our approach outperforms fourteen clustering methods on highly underdetermined problems (e.g., limited observations) as well as on large sample datasets, and yields interpretable embedding dendrograms. Thus our novel approach improves significantly on existing methods, and enables finding scalable and interpretable biomarkers for precision medicine.

Research studying the prediction of antibiotic resistance based on mass spectrometry data and machine learning focuses only on simple models for the identification of resistance to one antibiotic at a time, Even though a problem of multidrug resistance is currently being faced. Therefore, in this study, a multi-label approach for classifying multidrug resistance in Escherichia coli samples using raw MALDI-TOF mass spectrometry data and deep learning techniques was developed. The spectra from a recently published public database, encompassing over 4,500 samples of the bacteria under study, were utilized, sufficient for training a deep learning model, specifically a one dimensional convolutional neural network for this case. The use of this architecture proves to be highly efficient, achieving weighted AUROC and AUPRC values equal to or greater than 0.80, as well as a general performance calculated using the Hamming loss metric reaching 0.132. These results demonstrate that the use of deep learning allows for the development of complex models that enable the simultaneous identification of a predefined set of antibiotics, aiding in the determination of a highly effective treatment.

Mixed integer linear programs (MILP) are flexible and powerful tool for modeling and solving many difficult real-world combinatorial optimization problems. In this paper, we propose a novel machine learning (ML)-based framework ConPaS that learns to predict solutions to MILPs with contrastive learning. For training, we collect high-quality solutions as positive samples and low-quality or infeasible solutions as negative samples. We then learn to make discriminative predictions by contrasting the positive and negative samples. During test time, we predict assignments for a subset of integer variables of a MILP and then solve the resulting reduced MILP to construct high-quality solutions. Empirically, we show that ConPaS achieves state-of-the-art results compared to other ML-based approaches in terms of the quality of and the speed at which the solutions are found.

The automatic classification of non-functional requirements helps to reduce time and effort for the stakeholders. Several techniques have been used for this task, including the latest techniques in Machine Learning (ML) and Natural Language Processing (NLP), such as pre-trained models, with promising results. This research aims to analyze the performance metrics to classify requirements into sub-classes of non-functional type. Six distinct algorithms, including both traditional machine learning (ML) and deep learning (DL), were trained using a Spanish-translated PROMISE NFR dataset to assess and compare their performance outcomes. The findings reveal that SVM with BoW and fastText overperformed the other algorithms, however, fastText stands out between the two due the ease of implementation and the absence of data pre-processing.

General-purpose agents require fine-grained controls and rich sensory inputs to perform a wide range of tasks. However, this complexity often leads to intractable decision-making. Traditionally, agents are provided with task-specific action and observation spaces to mitigate this challenge. To address this, agents must be capable of building state-action spaces at the correct abstraction level from their sensorimotor experiences. We leverage the structure of a given set of temporally-extended actions to learn abstract Markov decision processes (MDPs) that operate at a higher level of temporal and state granularity. We characterize state abstractions necessary to ensure that planning with these skills, by simulating trajectories in the abstract MDP, is sufficient to derive policies with bounded value loss in the original MDP.We evaluate our approach in goal-based navigation environments that require continuous abstract states to plan successfully and show that abstract model learning improve the sample efficiency of planning and learning.

In recent times, social networks like Twitter have emerged as vital platforms for sharing personal thoughts, opinions, and most importantly, health-related information, especially pertaining to COVID-19. Users tend to share very detailed and personal narratives that could be utilized by researchers to capture true self-reported health data. While the data is easily accessible, the process to differentiate between health-related self-reports and informal discussion is quite tricky as it relies on either manual curation or the availability of large manually annotated datasets for machine learning models to be trained on. Manually annotating data is an immensely time-consuming task since, in general, the intervention of a subject matter expert is required, even more, in languages other than English, such as Spanish. In this work, we release two manually annotated datasets, one in English and one in Spanish, comprising of 36,548 tweets containing self-reported COVID-19 symptoms to aid machine learning models in extracting self-reported COVID-19 tweets. Using a very large set of experiments, we demonstrate how these datasets can be leveraged using classical and modern machine learning algorithms to identify unlabeled self-report tweets. Additionally, we perform a stratified analysis of how (and if) data augmentation and automatic translation could help train more generalizable models.

In the domain of machine learning, one persistent challenge is the availability of ample data, especially pertinent to computer vision. Moreover, this challenge is amplified within the realm of remote sensing, where annotations for addressing problems are frequently scarce. This manuscript critically examines the daunting task of monitoring a geophysical phenomenon —landslides— within the Peruvian landscape, a nation profoundly impacted by such events on a global scale. In this paper, we present three contributions in that direction. Our first contribution is to expand a well-known satellite imagery dataset targeting landslides. The nucleus of this foundational dataset originates from Asian territories, comprising 3799 meticulously annotated images. However, recognizing the distinct geospatial dynamics of Peru, we embarked on a rigorous exercise to augment this dataset with 838 local scenarios. These additions maintain congruence with the original dataset in terms of attributes and configuration, thereby ensuring both replicability and scalability for future research endeavours.Our second contribution is an exhaustive assessment of various semantic segmentation models. At the heart of our experimentation lies the U-net architecture, bolstered by the Weighted Cross Entropy + Dice Loss —a loss function acclaimed for its efficacy in segmentation tasks with imbalanced data sets. The empirical findings are illuminating: a rudimentary U-net architecture exhibits a formidable F1-Score of 75.5\%, transcending the benchmark score of 71.65\% set by the original dataset.Our third and final contribution is the comprehensive research framework developed for data acquisition, processing pipeline and model training/evaluation. Given this framework has the potential to drive a general applicability of segmentation systems to landslide monitoring systems, and to have a broader reach to the academic community and governmental stakeholders in Latin America and worldwide, we will be making all scripts and experiment details available upon publication, thus, hoping to foster an environment for collaborative scrutiny, discourse, and further advancement.

Accurate medical imaging segmentation is critical for precise and effective medical interventions. However, despite the success of convolutional neural networks (CNNs) in medical image segmentation, they still face challenges in handling fine-scale features and variations in image scales. These challenges are particularly evident in complex and challenging segmentation tasks, such as the BraTS multi-label brain tumor segmentation challenge. In this task, accurately segmenting the various tumor sub-components, which vary significantly in size and shape, remains a significant challenge, with even state-of-the-art methods producing substantial errors. Therefore, we propose two architectures, FMG-Net and W-Net, that incorporate the principles of geometric multigrid methods for solving linear systems of equations into CNNs to address these challenges. Our experiments on the BraTS 2020 dataset demonstrate that both FMG-Net and W-Net outperform the widely used U-Net architecture regarding tumor subcomponent segmentation accuracy and training efficiency. These findings highlight the potential of incorporating the principles of multigrid methods into CNNs to improve the accuracy and efficiency of medical imaging segmentation.

Anomaly detection, an essential component of industrial quality control and surveillance, plays a crucial role in identifying deviations from normal patterns. This extended abstract explores a preliminary research focused on enhancing the anomaly detection capabilities of the PaDim architecture—a state-of-the-art solution for anomaly detection on the MVTEC dataset—through the integration of Spatial Transform Networks (STNs). The aim is to improve performance, particularly in challenging classes such as "zipper" and "screw," where the PaDim architecture achieves lower metrics of performance. Notably, these challenging scenarios often involve objects in non-fixed positions, making anomaly detection intricate in real-world complex scenarios. Through experimentation involving the integration of a Spatial Transform Network using self-supervised training, the performance of this innovative approach is evaluated and it sheds light on both the strengths and limitations of this integration, providing insights into the benefits of leveraging Spatial Transform Networks to handle real-world complexity.

Drug repositioning, the identification of novel uses of existing therapies, has become an attractive strategy to accelerate drug development. Recently, knowledge graphs (KGs) have emerged as a powerful representation of interconnected data within the biomedical domain. While biomedical KGs can be used to predict new connections between compounds and diseases, most approaches only state \textit{whether} two nodes are related. Yet, they fail to explain \textit{why} two nodes are related. In this project, we introduce an implementation of the semi-parametric Case-Based Reasoning over subgraphs approach (CBR-SUBG), designed to derive the underlying mechanisms for a drug query by gathering graph patterns of similar entities. We show that our adaptation outperforms existing KG link prediction models on a drug repositioning task. Furthermore, our findings demonstrate that CBR-SUBG strategy can not only rank potential repositioning candidates but also provide interpretable biological paths, leading to more informed decisions.

This paper explores the sensitivity to changes in the image backgrounds of zero-shot image classifiers, such as those based on Large Language Models (LLMs) and Visual Language Models (VLMs). Specifically, we evaluate the image background robustness of VLM-only and the LLM+VLM image classifiers, verifying how background information influences their similarity scores and subsequently impacts their accuracy. For that analysis, we use the CLIP, ALIGN, ChatGPT+CLIP, and ChatGPT+ALIGN model for zero-shot image classification and compare its performance with baseline architectures such as Vision Transformer (ViT) and ResNet on the ImageNet-9 and RIVAL10 background challenges. The results indicate that all models exhibit some limitations when faced with background shifts; the ChatGPT+CLIP and CLIP-only models experienced a significant decrease in accuracy, suggesting difficulties performing accurate foreground-only and background-shift classification. However, the Align model consistently outperforms the other models in handling background variations. Our findings underscore the ongoing challenge of addressing background shifts in image classification and offer valuable insights for future improvements.

Tauopathies, a subset of neurodegenerative diseases, are anticipated to surge in incidence in the coming decades. While Machine and Deep Learning algorithms have significantly advanced medical imaging, the potential of post-mortem immunofluorescence imaging of brain tissues, particularly for monitoring Tau protein anomalies, remains largely untapped. This study presents a Convolutional Neural Network pipeline leveraging the ResNet-IFT architecture and Transfer Learning to classify Tau pathologies in Alzheimer's disease and Progressive Supranuclear Palsy using post-mortem immunofluorescence images. Among four tested architectures, our models consistently showcased an average accuracy of 98.41%, shedding light on the unique structural patterns of Tau protein in NFTs.

Statistical divergences quantify the difference between probability distributions, thereby allowing for multiple uses in machine-learning. However, a fundamental challenge of these quantities is their estimation from empirical samples since the underlying distributions of the data are usually unknown. In this work, we propose a divergence inspired by the Jensen-Shannon divergence which avoids the estimation of the probability density functions. Our approach embeds the data in an reproducing kernel Hilbert space (RKHS) where we associate data distributions with uncentered covariance operators in this representation space. Therefore, we name this measure the representation Jensen-Shannon divergence (RJSD). We provide an estimator from empirical covariance matrices by explicitly mapping the data to an RKHS using Fourier features. This estimator is flexible, scalable, differentiable, and suitable for minibatch-based optimization problems. Additionally, we provide an estimator based on kernel matrices without an explicit mapping to the RKHS. We provide consistency convergence results for the proposed estimator. Moreover, we demonstrate that this quantity is a lower bound on the Jensen-Shannon divergence, leading to a variational approach to estimate it with theoretical guarantees. We leverage the proposed divergence to train generative networks, where our method mitigates mode collapse and encourages samples diversity. Additionally, RJSD surpasses other state-of-the-art techniques in multiple two-sample testing problems, demonstrating superior performance and reliability in discriminating between distributions.

In machine learning, classification tasks serve as the cornerstone of a wide range of real-world applications. Reliable, trustworthy classification is particularly intricate in biomedical settings, where the ground truth is often inherently uncertain and relies on high degrees of human expertise for labeling. Traditional metrics such as precision and recall, while valuable, are insufficient for capturing the nuances of these ambiguous scenarios. Here, we introduce the concept of aberrant predictions, emphasizing that the nature of classification errors is as critical as their frequency. We propose a novel, efficient training methodology aimed at both reducing the misclassification rate and discerning aberrant predictions. Our framework demonstrates a substantial improvement in model performance, achieving a 20\% increase in precision. We apply this methodology to the less-explored domain of veterinary radiology, where the stakes are high but have not been as extensively studied compared to human medicine. By focusing on the identification and mitigation of aberrant predictions, we enhance the utility and trustworthiness of machine learning classifiers in high-stakes, real-world scenarios, including new applications in the veterinary world.

Autism Spectrum Disorder (ASD) is a group of complex neurodevelopmental disorders that affects about 1% of the world’s population, impacting the quality of life of not only the diagnosed individuals but also their communities. Early detection and intervention are paramount to limit its effect on a child's development, however overlap with other disorders and medical comorbidities make these tasks challenging. The present study explores the use of a novel multimodal, interpretable approach to characterize ASD children's behavior in a naturalistic environment. Spatial (real-time location tracking), audio and demographic data from children in a classroom setting are integrated and analyzed to identify traits potentially connected to ASD. Our findings point to the use of this type of approach as a potential tool for screening individuals in a naturalistic setting, allowing for further evaluation and, ultimately, earlier diagnosis by a clinician. Results show good classification performance and suggest vocalization, speech, proximity and certain movement-related features to be impacted in ASD.

Geological storage of CO$_2$ is a key climate change mitigation strategy. As important as location selection and injection planning is monitoring that the gas is contained for long periods of time. We introduce a fully 3D, deep learning-based approach for the joint inversion of time-lapse surface gravity and seismic data for reconstructing subsurface density and velocity models. The target application of this proposed inversion approach is the prediction of subsurface CO2 plumes as a complementary tool for monitoring CO2 sequestration deployments. Our joint inversion technique outperforms deep learning-based gravity-only and seismic-only inversion models, achieving improved density and velocity reconstruction, accurate segmentation, and higher R-squared coefficients. Future work will focus on validating our approach with larger datasets, simulations with other geological storage sites, and, ultimately, field data.

The pervasive issue of human trafficking has increasingly manifested through digital platforms, particularly in the form of textual online advertisements. Leveraging Natural Language Processing (NLP) for risk assessment in this domain has garnered significant attention. This study presents a comprehensive empirical evaluation of machine learning models fine-tuned for emotion and sentiment analysis tasks, specifically utilizing the \textit{BERT-Base Uncased} and \textit{DistilBERT} architectures. These models are rigorously compared against a baseline model, also fine-tuned on the \textit{BERT-Base Uncased} architecture, for the task of human trafficking risk prediction. Employing robust statistical methodologies, namely the Friedman and Nemenyi tests, we scrutinize the performance metrics of these models. Our findings indicate that while task-specific fine-tuned models exhibit promising results, they do not statistically outperform the baseline model in the human trafficking risk prediction task. This research not only contributes to the growing body of work in NLP applications for social good but also provides valuable insights for future research directions in the field.

Clustering algorithms of the Neural Gas (NG) type take into consideration the dissimilarities between prototypes in the original input space. It has been successfully applied in vector quantisation, topology creation as well as clustering. NG algorithms conventionally are based on the squared Euclidean distance or L2, which have several known setbacks (not robust to noise and outliers). Our goal is to introduce new NG clustering algorithms (on-line and batch) based on the L1 distance (robust to noise and outliers). We propose three Neural Gas algorithms based on the L1 distance using two different algorithms to find the optimal prototypes and compare them with another well-known clustering algorithm. Given the experiments performed, the proposed methods showed a competitive performance. Preliminary results indicate that research on Neural Gas algorithms based on L1 distance is promising.