Continuously Train + Deploy TensorFlow Serverless Models in Production (Kafka + OpenFaaS + Kubernetes + GPU)

Tue Apr 10
05:30 PM
Claim this listing


5:30pm Doors Open, Drinks, Food
6:15pm Doors Close (Must be here by this time!!)
6:30pm Talks Start
8:00pm Networking and Mingling
9:00pm Goodbye!
Continuous Training and Deploying of High Performance, Serverless TensorFlow Models in Production with Jupyter, TensorFlow, Scikit-Learn, Kafka, Kubernetes, Istio, OpenFaaS, Prometheus, Grafana, Slack, and GPUs
AbstractUsing the latest advancements in real-time AI from the open source PipelineAI project, I will demonstrate how to continuously train and deploy GPU-based TensorFlow models using live streaming data on a hybrid-cloud Kubernetes cluster.Streaming data is generated in real-time from the audience using a Slack to crowd-source the data labeling. This newly-labeled data automatically generates new model variants.I will use OpenFaaS and Istio with Kubernetes to quickly - and safely - deploy the new model variants to live production traffic.Similar to canary deployments of classic microservices, the new model variants are deployed safely to production in a controlled manner. Initially, they are exposed to only a small amount of traffic.Using reinforcement learning, multi-armed bandits, and metrics from Prometheus/Grafana, live traffic is automatically routed to the winning models based on a given reward function such as MAXIMIZE(number of signups) or MINIMIZE(cost per prediction).All demos run on a hybrid-cloud, open source, GPU-based, Kubernetes cluster optimized for the machine learning and artificial intelligence use cases that we commonly see at PipelineAI.BioChris Fregly is Founder and Applied AI Engineer at PipelineAI, a Real-Time Machine Learning and Artificial Intelligence Startup based in San Francisco. He is also an Apache Spark Contributor, a Netflix Open Source Committer, founder of the Global Advanced Spark and TensorFlow Meetup, author of the O’Reilly Training and Video Series titled, "High Performance TensorFlow in Production with Kubernetes and GPUs."Previously, Chris was a Distributed Systems Engineer at Netflix, a Data Solutions Engineer at Databricks, and a Founding Member and Principal Engineer at the IBM Spark Technology Center in San Francisco.


  1. Yelp 140 New Montgomery St , San Francisco, CA