Run Streamlit on Firecracker

Streamlit is an amazing library that allows one to build rich web applications in Python with a fantastic UI. Check out their website: https://www.streamlit.io/.

My first attempt was to install streamlit dependencies using native alpine packages. These dependencies are pretty heavy, and I increased rootfs size to 3GB.

$ apk add python3 python3-dev gfortran build-base py3-numpy 
$ apk add git libffi-dev zlib-dev libzmq zeromq-dev cmake
$ pip3 install numpy>=1.16.4
$ pip3 install streamlit

In the very end, I had to compile pyarrow from the source code, which was, to put it lightly, a frustrating experience. I followed instructions from https://gist.github.com/bskaggs/fc3c8d0d553be54e2645616236fdc8c6:

ARROW_SHA1=2ede75769e12df972f0acdfddd53ab15d11e0ac2 ARG
ARROW_BUILD_TYPE=release

ENV ARROW_HOME=/usr/local \
    PARQUET_HOME=/usr/local

#Download and build apache-arrow
RUN mkdir /arrow \
    && apk add --no-cache curl \
    && curl -o /tmp/apache-arrow.tar.gz -SL https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz \
    && echo "$ARROW_SHA1 *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz \
    && tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1 \
    && mkdir -p /arrow/cpp/build \
    && cd /arrow/cpp/build \
    && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \
	  -DCMAKE_INSTALL_LIBDIR=lib \
	  -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
	  -DARROW_PARQUET=on \
	  -DARROW_PYTHON=on \
	  -DARROW_PLASMA=on \
	  -DARROW_BUILD_TESTS=OFF \
	  .. \
    && make -j$(nproc) \
    && make install \
    && cd /arrow/python \
    && python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet \
    && python setup.py install \
    && rm -rf /arrow /tmp/apache-arrow.tar.gz

Depending on your environment, it may take some time to build everything, so pour yourself some hot liquid or watch a short YouTube video.

However, that’s a lot of libraries to install! I had to manually resolve all dependencies to figure out which packages it requires under the hood. Not fun. It must be an easy way how to do this, right?

I know that Conda maintains a solid repository, and thousands of volunteers help with the dependencies. I decided to give it a try.

I’m going to use continuumio/miniconda3 image as a base since it comes with a pre-built conda:

FROM continuumio/miniconda3:4.9.2-alpine

RUN apk add openrc
RUN apk add util-linux

# Set up a login terminal on the serial console (ttyS0):
RUN ln -s agetty /etc/init.d/agetty.ttyS0
RUN echo ttyS0 > /etc/securetty
RUN rc-update add agetty.ttyS0 default

# Make sure special file systems are mounted on boot:
RUN rc-update add devfs boot
RUN rc-update add procfs boot
RUN rc-update add sysfs boot

RUN echo 'root:root' | chpasswd

# streamlit and dependencies: numpy, pandas, pyarrow, etc
RUN apk add python3 py3-pip
RUN apk add dbus sudo
RUN . /root/.shinit
RUN conda install -y pyarrow
RUN pip3 install streamlit

Once an image is built, I have a lousy 600MB left on the device. Perhaps, later I can remove build dependencies and unneeded tests to reduce the size of the image.

The final container is taking 530MiB of virtual memory and 220MiB of resident memory. It’s less than ideal, but knowing what libraries I had to install, it feels reasonable enough.

Now, I can run streamlit inside a Firecracker.

# streamlit hello

  Welcome to Streamlit. Check out our demo in your browser.

  Network URL: http://172.16.0.2:8501
  External URL: http://xxx.xxx.xxx.xxx:8501

  Ready to create your own Python apps super quickly?
  Head over to https://docs.streamlit.io

  May you create awesome apps!