As the impact of data science & engineering reaches increasingly farther and wider, our professional responsibility as researchers and practitioners becomes more critical to society. The production data-driven systems we research, design, build and operate often bring inherent adversities with complex technical, societal and even ethical challenges. The skillsets required to tackle these challenges require us to go beyond the algorithms, and leverage diverse and cross-functional collaboration that often goes beyond a single data scientist or developer. In this talk we will cover practical insights from key core ethics themes in data science & engineering, including Privacy, Equity, Trust and Transparency. We will dive into the importance of these core themes, the growing societal challenges, and how organisations such as The Institute for Ethical AI, The Linux Foundation, the Association for Computer Machinery, NumFocus, the IEEE and other relevant academic and industry organisations are contributing to these through standards, policy advise and open source software initiatives.

Successful applications of deep reinforcement learning (deep RL) combine algorithmic design and careful hyper-parameter selection. The former often comes from iterative improvements over existing algorithms, while the latter is either inherited from prior methods or tuned for the specific method being introduced. Although critical to a method’s performance, the effect of the various hyper-parameter choices are often overlooked in favour of algorithmic advances. In this paper, we perform an initial empirical investigation into a number of often-overlooked hyper-parameters for value-based deep RL agents, demonstrating their varying levels of importance. We conduct this study on a varied set of classic control environments which helps highlight the effect each environment has on an algorithm’s hyper-parameter sensitivity.

We study the privacy limitations of label differential privacy, which has emerged as an intermediate trust model between local and central differential privacy, where only the label of each training example is protected (and the features are assumed to be public). We show that the guarantees provided by label DP are significantly weaker than they appear, as an adversary can "un-noise" the perturbed labels. Formally we show that the privacy loss has a close connection with Jeffreys' divergence of the conditional distribution between positive and negative labels, which allows explicit formulation of the trade-off between utility and privacy in this setting. Our results suggest how to select public features that optimize this trade-off. But we still show that there is no free lunch---instances where label differential privacy guarantees are strong are exactly those where a good classifier does not exist. We complement the negative results with a non-parametric estimator for the true privacy loss, and apply our techniques on large-scale benchmark data to demonstrate how toachieve a desired privacy protection.

Uncertainty quantification in neural network promises to increase safety of AI systems, but it is not clear how performance might vary with the training set size. In this paper we evaluate seven uncertainty methods on Fashion MNIST and CIFAR10, as we sub-sample and produce varied training set sizes. We find that calibration error and out of distribution detection performance strongly depend on the training set size, with most methods being miscalibrated on the test set with small training sets. Gradient-based methods seem to poorly estimate epistemicuncertainty and are the most affected by training set size. We expect our results can guide future research into uncertainty quantification and help practitioners select methods based on their particular available data.

We propose to approach the problem of learning $L_0$-sparse networks using a constrained formulation of the optimization problem. This is in contrast to commonly used penalized approaches, which combine the regularization terms additively with the (surrogate) empirical risk. Our experiments demonstrate that we can obtain approximate solutions to the constrained optimization problem with comparable performance to state-of-the art methods for $L_0$-sparse training. Finally, we discuss how this constrained approach provides greater (hyper-)parameter interpretability and accountability from a practitioner's point of view.

In recent years, researchers have started looking into data transformations in quantum computation. They want to see how quantum computing affects the robustness and performance of machine learning methods. Quantum mechanics succeed in explaining some phenomena where classical formulas failed in the past. Thus, it expanded in analytical research fields such as Quantum Machine Learning (QML) over the years. The developing QML discipline has proven solutions to issues that are equivalent (or comparable) to those addressed by classical machine learning, including classification and prediction problems using quantum classifiers. As a result of these factors, quantum classifier analysis has become one of the most important topics in QML. This paper studies four quantum classifiers: Support Vector Classification with Quantum Kernel (SVCQK), Quantum Support Vector Classifier (QSVC), Variational Quantum Classifier (VQC), and Circuit Quantum Neural Network Classifier (CQNNC). We also report case study outcomes and results analysis utilizing linearly and non-linearly separable datasets generated. Our research is to explore if quantum information may aid learning or convergence.

In modern days, social media platforms provide accessible channels for the inter-action and immediate reflection of the most important events happening around the world. In this paper, we, firstly, present a curated set of datasets whose origin stem from the Twitter’s Information Operations efforts. More notably, these accounts, which have been already suspended, provide a notion of how state-backed human trolls operate.Secondly, we present detailed analyses of how these behaviours vary over time,and motivate its use and abstraction in the context of deep representation learning:for instance, to learn and, potentially track, troll behaviour. We present baselinesf or such tasks and highlight the differences there may exist within the literature.Finally, we utilize the representations learned for behaviour prediction to classify trolls from"real"users, using a sample of non-suspended active accounts.

Pest insects are a problem in horticulture, so early detection is key for their control. Sticky traps are an inexpensive way to obtain insect samples in crops, but identifying them manually is a time-consuming task. Building computational models to identify insect species in sticky trap images is therefore highly desirable. However, this is a challenging task due to the difficulty in getting sizeable sets of training images. In this paper, we studied the usefulness of three neural network generative models to synthesize pest insect images (DCGAN, WGAN, and VAE) for augmenting the training set and thus facilitate the induction of insect detector models. Experiments with images of seven species of pest insects of the Peruvian horticulture showed that the WGAN and VAE models are able to learn to generate images of such species. It was also found that the synthesized images can help to induce YOLOv5 detectors with significant gains in detection performance compared to not using synthesized data. A demo app that integrates the detector models can be accessed through the URL: https://bit.ly/3uXW0Ee

Researchers have extensively used social media platforms like Twitter for knowledge discovery purposes, as tweets are considered a wealth of information that provides unique insights. Recent developments have further enabled social media mining for various biomedical tasks such as pharmacovigilance. A first step towards identifying a use-case of Twitter for the pharmacovigilance domain is to extract medication/drug terminologies mentioned in the tweets, which is a challenging task due to several reasons. For example, drug mentions in tweets may be incorrectly written, making the identification of these mentions more difficult. In this work, we propose a two step approach, first, we focused on classifying tweets with drug mentions via an ensemble model (containing transformer models), second, we extract drug mentions (along with their span positions) using a text-tagging/dictionary based approach, and a Named Entity Recognition (NER) approach. By comparing these two entity identification approaches, we demonstrate that using only a dictionary-based approach is not enough.

The Stochastic Streamflow Models (SSMS) are time series models for precise prediction of hydrological data useful in hydrologic risk management. Nowadays, deep learning networks get many considerations in time series forecasting. However, despite their theoretical benefits, they fail due to their drawbacks, such as complex architectures, slow convergence and the vanishing gradient problem. In order to alleviate these drawbacks, we propose a new stochastic model applied in problems that involve stochastic behavior and periodic characteristics. The new model has two components, the first one, a type of recurrent neural network embedding the echo-state (ESN) learning mechanism instead of conventional backpropagation. The last component adds the uncertainty associated with stationary processes. This model is called Stochastic Streamflow Model ESN (SSMESN). It was calibrated with time series of monthly discharge data from MOPEX data set. Preliminar results show that the SSMESN can achieve a significant prediction performance, learning speed. This model, can be considered a first attempt that applies the echo state network methodology to stochastic process.

Currently, the standard for vehicle speed estimation on urban areas and highways is radar or lidar speed signs which can be costly to buy, install and maintain. In addition, most major cities already implement networks of traffic surveillance cameras that can be utilized for vehicle speed estimation using computer vision. This work designs and implements such a system. Specifically, the proposed method is composed of three main components. First, we propose a camera calibration using homography with point correspondences between the image and world plane selected by the user. Second, a YOLOv4 object detector to locate the vehicles, and third, a modified object tracker that estimates vehicle speed. Moreover, for the calibration, a new method for camera calibration is developed specifically for this use case, using the estimation of density evolutionary algorithm. This methodology aims at correcting the misalignment between a point in the image plane and the world plane produced by the human operation. In addition, a basic direct linear transformation (DLT) and a random sample consensus robust version of DLT are implemented for comparison. Finally, the results show that the workflow using the proposed homography estimation method reduces the projection error from world to image point by 97\%, when compared to the other two methods, and the complete workflow can successfully register speed distributions expected from vehicles on urban traffic and handle dynamic changes in vehicle speed.

Successful applications of deep reinforcement learning (deep RL) combine algorithmic design and careful hyper-parameter selection. The former often comes from iterative improvements over existing algorithms, while the latter is either inherited from prior methods or tuned for the specific method being introduced. Although critical to a method’s performance, the effect of the various hyper-parameter choices are often overlooked in favour of algorithmic advances. In this paper, we perform an initial empirical investigation into a number of often-overlooked hyper-parameters for value-based deep RL agents, demonstrating their varying levels of importance. We conduct this study on a varied set of classic control environments which helps highlight the effect each environment has on an algorithm’s hyper-parameter sensitivity.

We study the privacy limitations of label differential privacy, which has emerged as an intermediate trust model between local and central differential privacy, where only the label of each training example is protected (and the features are assumed to be public). We show that the guarantees provided by label DP are significantly weaker than they appear, as an adversary can "un-noise" the perturbed labels. Formally we show that the privacy loss has a close connection with Jeffreys' divergence of the conditional distribution between positive and negative labels, which allows explicit formulation of the trade-off between utility and privacy in this setting. Our results suggest how to select public features that optimize this trade-off. But we still show that there is no free lunch---instances where label differential privacy guarantees are strong are exactly those where a good classifier does not exist. We complement the negative results with a non-parametric estimator for the true privacy loss, and apply our techniques on large-scale benchmark data to demonstrate how toachieve a desired privacy protection.

Uncertainty quantification in neural network promises to increase safety of AI systems, but it is not clear how performance might vary with the training set size. In this paper we evaluate seven uncertainty methods on Fashion MNIST and CIFAR10, as we sub-sample and produce varied training set sizes. We find that calibration error and out of distribution detection performance strongly depend on the training set size, with most methods being miscalibrated on the test set with small training sets. Gradient-based methods seem to poorly estimate epistemicuncertainty and are the most affected by training set size. We expect our results can guide future research into uncertainty quantification and help practitioners select methods based on their particular available data.

We propose to approach the problem of learning $L_0$-sparse networks using a constrained formulation of the optimization problem. This is in contrast to commonly used penalized approaches, which combine the regularization terms additively with the (surrogate) empirical risk. Our experiments demonstrate that we can obtain approximate solutions to the constrained optimization problem with comparable performance to state-of-the art methods for $L_0$-sparse training. Finally, we discuss how this constrained approach provides greater (hyper-)parameter interpretability and accountability from a practitioner's point of view.

The Stochastic Streamflow Models (SSMS) are time series models for precise prediction of hydrological data useful in hydrologic risk management. Nowadays, deep learning networks get many considerations in time series forecasting. However, despite their theoretical benefits, they fail due to their drawbacks, such as complex architectures, slow convergence and the vanishing gradient problem. In order to alleviate these drawbacks, we propose a new stochastic model applied in problems that involve stochastic behavior and periodic characteristics. The new model has two components, the first one, a type of recurrent neural network embedding the echo-state (ESN) learning mechanism instead of conventional backpropagation. The last component adds the uncertainty associated with stationary processes. This model is called Stochastic Streamflow Model ESN (SSMESN). It was calibrated with time series of monthly discharge data from MOPEX data set. Preliminar results show that the SSMESN can achieve a significant prediction performance, learning speed. This model, can be considered a first attempt that applies the echo state network methodology to stochastic process.

In modern days, social media platforms provide accessible channels for the inter-action and immediate reflection of the most important events happening around the world. In this paper, we, firstly, present a curated set of datasets whose origin stem from the Twitter’s Information Operations efforts. More notably, these accounts, which have been already suspended, provide a notion of how state-backed human trolls operate.Secondly, we present detailed analyses of how these behaviours vary over time,and motivate its use and abstraction in the context of deep representation learning:for instance, to learn and, potentially track, troll behaviour. We present baselinesf or such tasks and highlight the differences there may exist within the literature.Finally, we utilize the representations learned for behaviour prediction to classify trolls from"real"users, using a sample of non-suspended active accounts.

Pest insects are a problem in horticulture, so early detection is key for their control. Sticky traps are an inexpensive way to obtain insect samples in crops, but identifying them manually is a time-consuming task. Building computational models to identify insect species in sticky trap images is therefore highly desirable. However, this is a challenging task due to the difficulty in getting sizeable sets of training images. In this paper, we studied the usefulness of three neural network generative models to synthesize pest insect images (DCGAN, WGAN, and VAE) for augmenting the training set and thus facilitate the induction of insect detector models. Experiments with images of seven species of pest insects of the Peruvian horticulture showed that the WGAN and VAE models are able to learn to generate images of such species. It was also found that the synthesized images can help to induce YOLOv5 detectors with significant gains in detection performance compared to not using synthesized data. A demo app that integrates the detector models can be accessed through the URL: https://bit.ly/3uXW0Ee

In recent years, researchers have started looking into data transformations in quantum computation. They want to see how quantum computing affects the robustness and performance of machine learning methods. Quantum mechanics succeed in explaining some phenomena where classical formulas failed in the past. Thus, it expanded in analytical research fields such as Quantum Machine Learning (QML) over the years. The developing QML discipline has proven solutions to issues that are equivalent (or comparable) to those addressed by classical machine learning, including classification and prediction problems using quantum classifiers. As a result of these factors, quantum classifier analysis has become one of the most important topics in QML. This paper studies four quantum classifiers: Support Vector Classification with Quantum Kernel (SVCQK), Quantum Support Vector Classifier (QSVC), Variational Quantum Classifier (VQC), and Circuit Quantum Neural Network Classifier (CQNNC). We also report case study outcomes and results analysis utilizing linearly and non-linearly separable datasets generated. Our research is to explore if quantum information may aid learning or convergence.

Researchers have extensively used social media platforms like Twitter for knowledge discovery purposes, as tweets are considered a wealth of information that provides unique insights. Recent developments have further enabled social media mining for various biomedical tasks such as pharmacovigilance. A first step towards identifying a use-case of Twitter for the pharmacovigilance domain is to extract medication/drug terminologies mentioned in the tweets, which is a challenging task due to several reasons. For example, drug mentions in tweets may be incorrectly written, making the identification of these mentions more difficult. In this work, we propose a two step approach, first, we focused on classifying tweets with drug mentions via an ensemble model (containing transformer models), second, we extract drug mentions (along with their span positions) using a text-tagging/dictionary based approach, and a Named Entity Recognition (NER) approach. By comparing these two entity identification approaches, we demonstrate that using only a dictionary-based approach is not enough.

Currently, the standard for vehicle speed estimation on urban areas and highways is radar or lidar speed signs which can be costly to buy, install and maintain. In addition, most major cities already implement networks of traffic surveillance cameras that can be utilized for vehicle speed estimation using computer vision. This work designs and implements such a system. Specifically, the proposed method is composed of three main components. First, we propose a camera calibration using homography with point correspondences between the image and world plane selected by the user. Second, a YOLOv4 object detector to locate the vehicles, and third, a modified object tracker that estimates vehicle speed. Moreover, for the calibration, a new method for camera calibration is developed specifically for this use case, using the estimation of density evolutionary algorithm. This methodology aims at correcting the misalignment between a point in the image plane and the world plane produced by the human operation. In addition, a basic direct linear transformation (DLT) and a random sample consensus robust version of DLT are implemented for comparison. Finally, the results show that the workflow using the proposed homography estimation method reduces the projection error from world to image point by 97\%, when compared to the other two methods, and the complete workflow can successfully register speed distributions expected from vehicles on urban traffic and handle dynamic changes in vehicle speed.