Polyglot Notebooks: A practical introduction
Following the Jupyter model, the .NET-centered Polyglot notebooks combine Markdown and code. They allow multiple languages in the same notebook.
(Bild: jakkaje879/Shutterstock.com)
- Dr. Eike M. Hirdes
- Arthur Grot
Microsoft's Polyglot Notebooks are modelled after the interactive Jupyter Notebooks, but are .NET-centric. This article provides a practical introduction to Polyglot Notebooks and describes their basic concepts, such as setup, using different programming languages, sharing variables and displaying results, using an example. Limitations in use and further topics are also discussed.
What are Polyglot Notebooks?
Polyglot Notebooks allow users to enrich code with documentation (in Markdown format), output the results and visualize them. The code blocks can be executed individually directly from the notebook and even combined with each other. Unlike the Jupyter notebooks often used for Python, they are designed to run multiple languages within one notebook.
As a result, Polyglot notebooks offer a wide range of application possibilities in software development:
Setting up Polyglot in VS Code
To use Polyglot Notebooks, the extension must first be installed in the source code editor Visual Studio Code (VS Code). This requires at least the .NET SDK 7.0 and the Polyglot Notebooks extension from the Visual Studio Marketplace. The extension also automatically installs the required Jupyter extensions.
After installing the required dependencies, a notebook can be created using the command palette in VS Code (Ctrl+Shift+P) and the command "Polyglot Notebook: Create default notebook" command to create a notebook. In this example, the .dib format is selected first. An explanation of the different file formats with their advantages and disadvantages will follow later. After selecting the format, a notebook opens with ".NET Interactive" as the selected kernel (Figure 1). The kernel is a service running in the background that executes the code from the notebook and returns the result as output. Each language has its own subkernel, which is responsible for executing the respective language.
Overview of Polyglot
The header bar not only displays the kernel, but also makes it possible to create additional code and markdown cells. A cell is a structural unit in the notebook that contains either code of a specific language or Markdown. This makes it possible to write code and Markdown documentation alternately (Figure 2).
The "Run All" command executes all cells in the notebook one after the other and displays the output. If problems occur during execution and the kernel gets into an incorrect state due to an error, it can be restarted. In the "Variables" menu, you can activate the view with which all variables in the kernel can be viewed. Each code cell can also be executed individually via the "Play" symbol (▷).
Polyglot's multi-language support currently includes the following programming and markup languages: C#, F#, PowerShell, JavaScript, SQL, KQL, HTML, Mermaid, Python and R. VS Code offers syntax highlighting, code autocompletion and other features for the languages. Since December 2023, Polyglot has also supported the HTTP protocol, which means that HTTP requests can be sent directly from notebooks and the responses displayed.
Polyglot Notebooks explained using an example
To demonstrate the features of Polyglot Notebooks, an example is used that is created from scratch. The initial situation is to quickly find out which customers have bought a certain product for a recall campaign. This data is stored in an SQL database. The SQL connection string required for this is to be set up in the first step and the access data queried for this. In the next step, a connection is established with the database and then the required data is retrieved. In the last step, the data is displayed as a diagram to quickly recognize which customer is affected and to what extent.
The example shows how different languages and technologies (Polyglot Magic Commands, C#, SQL and Mermaid, which creates graphics from Markdown) can be used in a notebook and how they interact with each other. Magic Commands are special commands for Polyglot notebooks that make it possible to execute code in other languages within a code cell or to control the notebook environment, for example to set variables.
Creating the SQL connection string - processing user input and variables
Once Polyglot has been set up, the connection string for the SQL database is created. This requires four parameters: the URL of the SQL server, the name of the database, a username and the password. The first two parameters are fixed, the last two should be entered by the user.
Markdown can be used to explain both the entire notebook and individual code cells. For this connection string, variables should be permanently defined, but users should also be asked for access data. The following cell in Markdown format is used for documentation and instructions:
## Create SQL Connection String
Run the following cell to create a sql connection string.
You will be prompted to input username and password.Next, create a C# cell to permanently define the URL of the SQL server (serverUrl) and the name of the database (databaseName) as variables. The username (username) and the password (password) are to be queried. The implementation for the query with Polyglot Magic Commands can be found in Listing 1. After calling the Magic Commands in the C# cell, the created variables can be used in the C# kernel.
// Fetch user input with magic commands
var serverUrl = "polyglot.database.windows.net";
var databaseName = " polyglot";
#!set --value @input:"Please provide a username" --name username
#!set --value @password:"Please provide a password" --name passwordListing 1: Querying user data with Magic Commands in a C# cell
If you prefer to implement this completely in C#, you must first integrate the Microsoft.DotNet.Interactive library, here as a NuGet package. The C# alternative can be seen in Listing 2.
// Fetch user input with C#
using Microsoft.DotNet.Interactive;
var serverUrl = "polyglot.database.windows.net";
var databaseName = "polyglot";
var username = await Kernel.GetInputAsync("Please provide a username");
var password = await Kernel.GetPasswordAsync("Please provide a password");LIsting 2: Querying user data in a C# cell
Both types of input support "Input" and "Password". With "Input" the input is displayed in plain text, whereas with "Password" it is masked. Note: Only the password input is masked, the content is visible in the variable view of Polyglot (Figure 3).
The SQL connection string is composed of the previously defined variables:
string connectionString = $"Server={serverUrl};Database={databaseName}; User
Id='{username}';Password='{password}';Persist Security Info=true;Integrated Security=false;";