We are excited to announce the availability of sparklyr.sedona, a sparklyr extension making geospatial functionalities of the Apache Sedona library easily accessible from R.continue reading.
Sparklyr 1.7 delivers much-anticipated improvements, including R interfaces for image and binary data sources, several new spark_apply() capabilities, and better integration with sparklyr extensions.continue reading.
The sparklyr 1.6 release introduces weighted quantile summaries, an R interface to power iteration clustering, spark_write_rds(), as well as a number of dplyr-related improvements.continue reading.
Unlike all three previous sparklyr releases, the recent release of sparklyr 1.5 placed much more emphasis on enhancing existing sparklyr features rather than creating new ones. As a result, many...continue reading.
We are excited to announce a number of powerful, new functionalities and improvements which are now part of sparklyr.flint 0.2!continue reading.
Sparklyr 1.4 is now available! This release comes with delightful new features such as weighted sampling and tidyr verbs support for Spark dataframes, robust scaler for standardizing data based on...continue reading.
We are pleased to announce that sparklyr.flint, a sparklyr extension for analyzing time series at scale with Flint, is now available on CRAN. Flint is an open-source library for working...continue reading.
How can the seemingly iterative process of weighted sampling without replacement be transformed into something highly parallelizable? Turns out a well-known technique based on exponential variates accomplishes exactly that.continue reading.
Sparklyr 1.3 is now available, featuring exciting new functionalities such as integration of Spark higher-order functions and data import/export in Avro and in user-defined serialization formats.continue reading.
sparklyr 1.3 is now available on CRAN, with the following major new features: Higher-order Functions to easily manipulate arrays and structs Support for Apache Avro, a row-oriented data serialization framework...continue reading.
A new version of sparklyr is now available on CRAN! In this sparklyr 1.2 release, the following new improvements have emerged into spotlight: A registerDoSpark() method to create a foreach...continue reading.
A new sparklyr release is now available. This sparklyr 1.2 release features new functionalities such as support for Databricks Connect, a Spark backend for the ‘foreach’ package, inter-op improvements for...continue reading.
Data Science / Machine learning / Python / R News / Spark
Data engineering functions on large datasets in Microsoft Fabric
September 25, 2024
GIS / R / R News / Shiny / Visualization
Interactive visualization of geospatial data with R Shiny
April 17, 2021
