Overview
Putki provides a commercial, enterprise-focused solution built on the Apache Hop open-source framework for data workflows. The platform addresses production challenges including stability, scalability, security, and support for organizations running data integration at scale. Putki bridges open-source software capabilities with enterprise-grade reliability and performance requirements.
In the news
- 𝗔𝗽𝗮𝗰𝗵𝗲 𝗛𝗼𝗽 𝟮.𝟮𝟬 𝗶𝘀 𝗼𝘂𝘁! 30 contributors, including 14 first-time contributors to Hop. → Native 𝗜𝗰𝗲𝗯𝗲𝗿𝗴 read/write. → 𝗔𝗜 𝘁𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝘀 for chunking, embeddings, and pgvector search, built directly into the pipeline. → 𝗛𝗼𝗽 𝗹𝗶𝗻𝘁, a new linting engine for pipelines and workflows. It runs in the GUI and the CLI, and can output SARIF for CI. → Pluggable graph database support for Neo4j, Memgraph, 𝗔𝗺𝗮𝘇𝗼𝗻 𝗡𝗲𝗽𝘁𝘂𝗻𝗲, 𝗙𝗮𝗹𝗸𝗼𝗿𝗗𝗕, 𝗔𝗽𝗮𝗰𝗵𝗲 𝗔𝗚𝗘, 𝗮𝗻𝗱 𝗚𝗿𝗲𝗺𝗹𝗶𝗻. → A new
- 𝗙𝗿𝗼𝗺 𝗦𝗦𝗜𝗦 𝘁𝗼 𝗔𝗽𝗮𝗰𝗵𝗲 𝗛𝗼𝗽 𝗧𝗼𝗽: an 𝗦𝗦𝗜𝗦 Data Flow reconstructed from the .dtsx package. 𝗕𝗼𝘁𝘁𝗼𝗺: the native Apache Hop pipeline generated by our importer. Moving away from a platform is not only about choosing a new platform. It is about understanding what happens to the ETL logic you already have. Our importer converts existing 𝗦𝗦𝗜𝗦 packages into native Apache Hop pipelines, giving teams a starting point for migration and visibility into what can be moved automatically and what needs attention.
- A few things worth highlighting from this week's Apache Hop activity: 🔹 Native Spark on YARN: Spark pipelines can now run directly on YARN clusters from both the Hop GUI and hop-run. 🔹 Pipeline hop graph exposed in lineage events: Lineage metadata now includes the actual hop graph, providing downstream lineage tools with more context about how pipelines are structured, not just which transforms were executed. 🔹 Git perspective performance improvements: Opening a project or switching editors no longer triggers a full repository git
- In January 2024, 𝗤𝗹𝗶𝗸 retired the free 𝗧𝗮𝗹𝗲𝗻𝗱 𝗢𝗽𝗲𝗻 𝗦𝘁𝘂𝗱𝗶𝗼. The jobs built with it still run, but the tool behind them no longer receives updates or security fixes. For some teams, the question is no longer if they should move, but when. The difficult part of a migration is usually not choosing a new platform. It is understanding what you have today. Years of jobs, created by different people, often with knowledge that is no longer available. Without that visibility, estimating the effort and cost of a migration
- 𝗔 𝗳𝗲𝘄 𝗵𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 𝗳𝗿𝗼𝗺 𝘁𝗵𝗶𝘀 𝘄𝗲𝗲𝗸: 🔹 Text File Input can now read files in parallel. Large files can be split across multiple reader threads, improving throughput for high-volume text ingestion. 🔹 Table Input and Database Join now properly stop running SQL queries when a pipeline is cancelled. 🔹 Filtered row viewer added to previews. You can now filter preview data while debugging transforms with large outputs. 🔹 There were also some important fixes behind the scenes: a security issue with
- Apache Hop, un ETL agile et open source pour motoriser vos projets d'intégration de données ! Accedez au replay de notre live talk : https://lnkd.in/ezB5UeVU Atol CD Sylvain Decloix
- Another big week for Apache Hop. The biggest highlight was the addition of samples for every transform and every action, giving new users a complete set of working examples right out of the box. Other highlights include Apache Iceberg support, #GraphRAG, the ability to run the Linter as a workflow action, a new Revisions tab in the GUI, SMB VFS connections, new #Beam IO connectors (including MQTT, Debezium, #Elasticsearch, Splunk, and Snowflake), and pluggable graph database connections with FalkorDB support. There were also plenty
- Sep 27 – Oct 4. 62 PRs, 82 issues closed, 5 contributors. 🔹 𝗣𝗜𝗜 𝗳𝗶𝗲𝗹𝗱 𝗺𝗮𝘀𝗸𝗶𝗻𝗴 landed. Reusable masking patterns you can apply across pipelines instead of building them from scratch. 🔹 𝗖𝗮𝗹𝗹 𝗗𝗕 𝗣𝗿𝗼𝗰𝗲𝗱𝘂𝗿𝗲 can now return result rows. 🔹 #Git repositories can now be accessed through a named #VFS connection, and an 𝗦𝗠𝗕 𝗩𝗙𝗦 𝗱𝗿𝗶𝘃𝗲𝗿 is also on the way. 🔹 Projects can now be searched from the project menu and grouped by topic. 🔹 𝗣𝗮𝗿𝗾𝘂𝗲𝘁 𝗢𝘂𝘁𝗽𝘂𝘁 gained logical field type support, and
Something wrong or missing? Send an update. Fixed within 24 hours.