Dask is a Python library for parallel and distributed computing. Dask is: Easy to use and set up (it’s just a Python library) Powerful at providing scale, and unlocking complex algorithms. and Fun 🎉

Dask is a flexible open-source Python library for parallel computing maintained by OSS contributors across dozens of companies including Anaconda, Coiled, SaturnCloud, and nvidia.

Dask provides ways to scale Pandas, Scikit-Learn, and Numpy workflows more natively, with minimal rewriting. It integrates well with these tools so that it copies most of their API and uses their data structures internally.

Creating a Dask Object¶ You can create a Dask object from scratch by supplying existing data and optionally including information about how the chunks should be structured.

Get inspired by learning how people are using Dask in the real world today, from biomedical research and earth science to financial services and urban engineering.

A Dask DataFrame is a large parallel DataFrame composed of many smaller pandas DataFrames, split along the index. These pandas DataFrames may live on disk for larger-than-memory computing on a single machine, or on many different machines in a cluster.

Welcome to the Dask Tutorial¶ Dask is a parallel and distributed computing library that scales the existing Python and PyData ecosystem. Dask can scale up to your full laptop capacity and out to a cloud cluster.

Dask for Machine Learning¶ This is a high-level overview demonstrating some the components of Dask-ML. Visit the main Dask-ML documentation, see the dask tutorial notebook 08, or explore some of the other machine-learning examples.

Dask works well at many scales ranging from a single machine to clusters of many machines. This page describes the many ways to deploy and run Dask, including the following: Python API. Cloud. High Performance Computers. Kubernetes

This installs Dask and all common dependencies, including pandas and NumPy. Dask packages are maintained both on the defaults channel and on conda-forge. You can select the channel with the -c flag: