Splitgraph – Build and Share Data with Postgres

Splitgraph – Build and Share Data with Postgres

sgr build votes.splitfile Work with data like you work with code. Try it in five minutes FROM splitgraph/uk_2017_ge:latest IMPORT { SELECT ons_code, SUM(CASE WHEN party_id = ‘Conservative’ THEN valid_votes ELSE 0 END) AS conservative_votes, SUM(CASE WHEN party_id = ‘Labour’ THEN valid_votes ELSE 0 END) AS labour_votes, SUM(valid_votes) AS total_votes FROM ward_results GROUP BY ons_code } AS votes_by_party SQL { CREATE TABLE london_votes AS SELECT lookup.”PCON18NM” AS constituency, v.conservative_votes, v.labour_votes, v.total_votes, ST_Union(london.geom) AS geom FROM “splitgraph/london_wards:latest”.city_merged_2018 london JOIN “splitgraph/uk_wards”.lookup_table lookup ON london.gss_code = lookup.”WD18CD” JOIN votes_by_party v ON v.ons_code = lookup.”PCON18CD” GROUP BY constituency, conservative_votes, labour_votes, total_votes } Build, combine and share data. Powered by Postgres. Inspired by Docker and Git. Build composable datasets Splitfiles allow you to use familiar SQL to build versioned datasets, or “data images,” which are snapshots of a database similar to how a Docker image is a snapshot of a filesystem. Merging public data with internal datasets is as simple as referencing them through a JOIN. Learn more about Splitfiles Keep data fresh and reproducible With Splitgraph’s provenance tracking, you know exactly where your data came from. Keep data images up-to-date with a single command when the sources change. Easily integrate Splitgraph into your CI pipeline to stay on top of changes to your data sources. Learn more about data provenance and rebuilding data images. Share data with peers Like Git, Splitgraph is peer-to-peer. Push data to any other Splitgraph instance or publish it to the catalog at Splitgraph Cloud, where you get bonus features like an instant, OpenAPI-compatible REST API for every version of your data. Learn more about Splitgraph Cloud. Try it in five minutesRead our introductory blog post » Keep your existing tools Anything that works with Postgres will work with Splitgraph. As far as your tools are concerned, a Splitgraph image is just another Postgres database. You can adopt Splitgraph incrementally while keeping your existing workflows and benefitting from the Postgres ecosystem. See examples of common integrations Ingest data from anywhere Forget ETL and BI connectors. Splitgraph leverages the native PostgreSQL feature of Foreign Data Wrappers. Use any FDW to import data from common databases, or thousands of open government datasets. Or, write a custom mount handler to import data from wherever you need. Read more about ingesting data with FDWs Save on costs Stop paying for a bulky, always-on data warehouse. Splitgraph data can be stored in any S3-compatible object storage and downloaded on demand when it needs to be queried. Read about layered querying Explore Public Data Explore over 40,000 datasets » Enhance every stage of the data lifecycle Adopt Splitgraph incrementally, where and when you need it. Ingest data from anywhereImport data from all major databases, setup Splitgraph as a Postgres replication client, or write a custom mount handler to cover your unique use case. Transform the data into a Splitgraph image, or leave it as-is and query it on demand.Read the FDW Documentation Layered queryingDon’t download the whole dataset just to run one SELECT. Splitgraph lets your software query remote data by lazily downloading only the required fragments.Learn about Layered Querying Instantly access thousands of open datasetsSplitgraph comes bundled with a mount handler for Socrata,  » Read More

Like to keep reading?

This article first appeared on splitgraph.com. If you'd like to keep reading, follow the white rabbit.

View Full Article

Leave a Reply