News from the sparkly-verse
Highlights to the most recent updates to `sparklyr` and friendscontinue reading.
Category: Spark
Advent of 2023, Day20 – Working with notebooks in Fabric
In this Microsoft Fabric series: Notebooks have been around for a long time and people, community, and professionals have proven the usability, practicality, versioning and reliability of notebooks. Not to...continue reading.
Advent of 2023, Day 14 – Data warehouse with Microsoft Fabric
In this Microsoft Fabric series: Today we will start exploring the Fabric Data Warehouse. With the data lake-centric logic, the data warehouse in Fabric is built on a distributed processing...continue reading.
Advent of 2023, Day 13 – Creating ML Model with Microsoft Fabric
In this Microsoft Fabric series: Yesterday, we created an MLFlow experiment and today we will look into creating an ML model. Create a new ML model, and give it a...continue reading.
Advent of 2023, Day 12 – Creating data science experiments with Microsoft Fabric
In this Microsoft Fabric series: We have started working with the data and now, we would like to create and submit the experiment. In this case, MLFlow will be used...continue reading.
Advent of 2023, Day 11 – Starting data science with Microsoft Fabric
In this Microsoft Fabric series: We have looked into creating the lakehouse, checked the delta lake and delta tables, got some data into the lakehouse, and created a custom environment...continue reading.
Comparing performances of CSV to RDS, Parquet, and Feather file formats in R
From the previous blogpost:– CSV or alternatives? Exporting data from SQL Server data to ORC, AVRO, Parquet, Feather files and store them into Azure data lake we have created Azure blob...continue reading.
SQL vs. NoSQL for Data Science
Data come in variety of form, at different pace, and at different volume. And if all three criteria define the difference between SQL and NoSQL and there, all three are...continue reading.
Advent of 2021, Day 25 – Spark literature, documentation, courses and books
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 24 – Data Visualisation with Spark
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 23 – Delta live tables with Azure Databricks
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 22 – Spark in Azure Databricks
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 21 – Spark GraphX operators
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 20 – Spark GraphX processing
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 19 – Data engineering for Spark Streaming
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 18 – Time windows for Spark streaming
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 17 – Watermarking and joins for Spark streaming
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 16 – Dataframe operations for Spark streaming
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 14 – Introduction to Spark Streaming
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
Advent of 2021, Day 14 – Spark SQL query hints and executions
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode...continue reading.
