Open-source data analytics project Ibis will get Voltron’s help

Ibis, an open source Python framework for standardising analytics and queries across many backends, has been added as an enterprise support solution by Voltron Data, a supplier of Apache Arrow corporate support services. Voltron Data is expanding its line of corporate support services with this move, which furthers the company’s commitment to open source technology. A virtual event sponsored by Voltron Data to showcase Apache Arrow and its practical applications as well as bring together practitioners, industry leaders, and members of the Arrow community is where this news was made.

The process of establishing norms

Josh Patterson, CEO of Voltron Data, talked with VentureBeat on the significance of setting standards and communication across systems to minimise pain points and lessen the amount of work that has to be put into each and every project.

Each of Apache’s offerings, Apache Arrow and Apache Ibis, is meant to address common pain points. Rather than having to serialise and deserialize data across several forms, Apache Arrow is an open source project that provides a single format for in-memory storage of vector data. In Patterson’s words, “simply allowing developers to concentrate on topics that they want to focus on” is the result of eliminating that duplication of effort.

Using the help of the Ibis Python dataframe API, you can easily obtain data from a variety of sources and perform complex operations on it with Python. Big data analytics databases such as Google BigQuery, (previously OmniSci), PostgreSQL, MySQL, and others are all supported. It is possible to design a single piece of code that can run on a variety of backends, all while maintaining the same foundation, thanks to the Ibis programming language. Users will no longer have to rewrite their code if the manner they do computations changes, according to Patterson.

To construct better and better systems, Patterson said that more people are building on these standards like Arrow [and] Ibis. As systems evolve, serialising data and rewriting code are two major sources of waste in the ecosystem. “These standard front-ends are critical if we are to eliminate these barriers to innovation,” says Xu.

Extending from the original idea

In February of 2022, Voltron Data was introduced with an Apache Arrow enterprise support package. Apache Arrow, Ibis, Python pandas and Apache Parquet project management committee member Wes McKinney are all co-creators of Voltron Data’s co-founder Wes McKinney. To some extent, Ibis’ business support was prompted by client demand, says the firm. Customers are wondering, “Well, what about Ibis?,” Patterson remarked. As a consequence, “we felt that it was a logical extension” to have the same individuals managing these initiatives.

How fast is the lift moving?

The Apache Arrow and Ibis benefits and efficiency come from their ability to give a single solution to users’ problem areas. In the case of Pandas and its dataframe paradigm, adoption of a standard tends to increase its value. Voltron’s decision makes logical and should help standardise data-oriented software development rather than fragment it, but it is unclear if Ibis will reach equal acceptance and value.