Distributed data processing: open source project ray 1.0 offers serverless api

Distributed Data Processing: Open Source Project Ray 1.0 offers Serverless API

The start-up anyscale has published the first major version of his project ray. Ray 1.0 is a framework for distributed programming, which is to be scalable for complex, roughly dimensioned data processing tasks, as they will apply for quantum computing for example. Roughest innovations are a stable API for serverless features and the integration of multiple libraries from the ray community (such as huggingface, intel analytics zoo and mars) into the okosystem of the framework. Ray 1.0 has native support for java on board and can be installed under windows.

Scalable okosystem for distributed data processing

Originally, the anyscale developers ray had designed as standard for distributed computing environments. Over the past three years, the project then developed into an extended okosystem with a series of libraries. The initiation of the first major major release took place on the first ray summit, which the start-up of the 30. September to 1. October 2020 had aligned virtual. In parallel, anyscale has introduced the private beta of a managed ray platform.

With ray 1.0 can be used to create highly scalable libraries, applications and services. The focus of distributed applications lies according to the blog entry on resources that run portable across various types of machine types, clusters and clouds. Automatically scaling over an API that controls the placement of resource requirements for suitable nodes, users should design the distributed data processing. The memory management can be automated, and the memory usage and application performance can be used and debug users in the dashboard.

The head behind the project

Behind the project is a four-headed team of the universitat berkeley, the computer science professor michael I. Jordan and the developer ion stoica accompanied. Among other things, stoica is co-generous of databicks and one of the original developers of apache spark. One of the main sponsors is apparently intel IT, but also companies like amazon, microsoft and ant financial seem to be interested in the start-up.

Further information can be found in the acquisition of the major version of ray in the anyscale blog.