Project Laminar

Schwab Intelligent Portfolios had a rough year in 2022. We did slightly better with the Laminar project, our implementation of a portfolio rebalancing bot which runs on the Alpaca platform and trades with real money.

For obvious reasons, the code will not be open source at any time soon. But this post shares how it works at a high level.

The entire project is composed of Luigi tasks, which allows us to build a data pipeline with complex dependencies that are automatically resolved. This helps us reason about the system and make sure there is no leakage of future data into our models.

Our data flow diagram looks something like this:

T = time

IngestNYT(T)
IngestOHLC(T)

(IngestNYT(0...T-1), IngestOHLC(0...T-1)) -> TrainModel(T)

IngestOHLC(T-1) -> EstimateCovariance(T)
(IngestNYT(T-1), IngestOHLC(T-1), TrainModel(T)) -> PredictReturns(T)

(EstimateCovariance(T), PredictReturns(T)) -> OptimizePortfolio(T)
OptimizePortfolio(T) -> RunLaminar(T)

• IngestNYT/IngestOHLC - These tasks download data from external APIs.
• TrainModel(T) - This trains a model using only data from T-1 and earlier. It uses Luigi's dependency resolution to figure out how to get the appropriate data so we don't have to reason about whether future data is leaking into our model.
• EstimateCovariance(T), PredictReturns(T) - These estimate the expected returns and covariance of returns.
• OptimizePortfolio(T) - This uses (mu, sigma) to generate several Pareto-efficient portfolios with varying levels of risk and then selects between them based on market conditions and sentiment extracted from the NYTimes.
• RunLaminar(T) - This examines the current portfolio on Alpaca (using real money!) and figures out how to achieve the new optimal portfolio.

At the end of the day, we simply call RunLaminar(Now()) with the current timestamp and it backfills all the dependencies as necessary, using cached results whenever possible.

We build a Docker container containing all the dependencies and push it to AWS Batch, which then schedules a job every Monday, Wednesday, and Friday to rebalance our portfolio. After every rebalancing, the system sends us an automated email with the current portfolio so we can do a quick sanity check.

This AWS job has been running on this schedule for over 4 months now and only costs tens of cents a month, so we plan to keep it running for the foreseeable future.

References

This project was powered by:

• Alpaca. Commission free trading API.
• Sendgrid. Easily send email notifications.
• Luigi. Lightweight library for building Python data pipeline.
• PyPortfolioOpt. Efficient implementation of common portfolio optimization strategies.
• Scikit-learn, PyTorch, and other standard ML libraries.