Data Science & Tech Blog

Communicating between Go and Python or R

By Matthew Mahowald on Thu, Jun 13, 2019

Data science and engineering teams at Open Data Group are polyglot by design: we like to choose the best tool for the task at hand. Most of the time, this means our services and components communicate through things like client libraries and RESTful APIs. But sometimes, we need code from one language to call code written in another language directly. In this post, we’ll take a short look at how to do that using C foreign function interfaces (FFI) as a way to call functions written in Go using Python.

Tags: python, R, golang

An Introduction to Hierarchical Models

By Steve Avsec on Thu, May 23, 2019

In a previous post we gave an introduction to Stan and PyStan using a basic Bayesian logistic regression model. There isn’t generally a compelling reason to use sophisticated Bayesian techniques to build a logistic regression model. This could be easily replicated using simpler techniques. In this post, we shall really unlock the power of Stan and full Bayesian inference in the form of a hierarchical model. Suppose we have a dataset which is stratified into N groups. We have a couple of choices for how to handle this situation. We can effectively ignore the stratification and pool all of the data together and train a model on all of the data at once. The cost of this is that we are losing information added by the stratification. We can fit a separate model for each group, but this runs the risk of overfitting. As we shall see below, groups with few observations will typically represent outliers, but this is not indicative that all observations in the group will behave the same way. If some of the groups have few samples and have significant outlying behavior, it is likely that the behavior is driven by the small sample size rather than the group exhibiting behavior that deviates significantly from the mean behavior.

Tags: bayesian, modeling, hierarchical

Machine Learning and Technological Explosion

By Nan Shi on Thu, May 09, 2019

In recent years, as commercial capital nourishing technology development, many technologies that challenge humanity are subverting human society: internet giant monopoly, DNA editing, artificial intelligence, etc. One of the core reason that made this “technological explosion” possible is the well-developed computer science industry, which revolutionized the inheritance of knowledge for human beings. More and more STEM elites are dedicating their whole life to the contemporary technological explosion, hoping to promote humanity to a whole new level. The rapid technological development naturally reminds me of my favorite science fiction “The Three-Body Problem”, where I first learned the concept of technological explosion:

Tags: machine learning