What’s New in Python 3.11? Most Important Changes

Hey there! My name is Oleksii. I am a 4th-year student at the Kyiv Polytechnic Institute and I’ve been programming in Python for around four years. Today, I’d like to update the column about #mostimportant_features that appear with new versions of Python and why you should use them on your projects 🙂.

But before jumping into the topic, if you haven’t read what's new in Python 3.10 yet, I’d recommend you do that, as there I’m talking about moving from Python 3.6 to Python 3.10, and why moving to newer Python versions is beneficial for your project 😉.

New features in Python 3.11

Release Date: October 2022

Exact error locations in tracebacks

It became much easier to detect errors, as now in the traceback the interpreter points not only to the line of code where the error occurs but to its exact place. A small change that will save you a lot of time. These extended errors can also be useful when dealing with deeply nested dict objects and multiple function calls.

Here’s what it looks like in Python 3.10:

And now, Python 3.11:

For better understanding, let’s create a random dictionary:

Now, let's try to refer to a non-existent key in one of the nested dictionaries.

Here’s a Python 3.10 example:

Python 3.11:

Exception groups and except*

In Python 3.11, it became possible to raise and handle multiple exceptions simultaneously. The ExceptionGroup and BaseExceptionGroup built-in types allow you to group exceptions and throw them together, while the new except* syntax generalizes exceptions to match subsets of exception groups.

Here’s an example from PEP-654:

But what about the except* syntax? As written in the PEP, the asterisk is an indicator that each group of exceptions can be caught with except* in one go.

Here’s an example of using except* with the same group of exceptions:

But I’d like to mention here that exception groups are a rather specific addition to the base exception classes, which already existed in many libraries and frameworks. The only reason one may have to use them is to receive exceptions simultaneously, which is a rare-case scenario for ordinary developers. Moreover, exception groups make the code more complicated, while there aren’t many situations where their addition is justified. So, when possible, I’d recommend not using them.

add_note for BaseException

Next up, I’m going to talk about another change in exceptions. For cases where the exception needs to be expanded with a note and cannot be done when the exception is initially created, an add_note() method was added to the BaseException base class. In __notes__ you can find a list of records added to the exception.

For example:

Required[] and NotRequired[] in TypedDict

A marking of optional fields was added to TypedDict, which was previously only possible through inheritance. The total=True parameter typically indicates that the fields are required, and in order to mark a specific field as optional, we had to create a second TypedDict in which we had to inherit the first one with all fields marked as required to then declare the new optional fields:

But now we have two new fields - Required and NotRequired (not to be confused with Optional, because Optional is Union[type, None], which is not the absence of a field, but the initial value).

For example:

LiteralString

When talking about changes in annotations, a new type - LiteralString - was added. It should be used to indicate that the function parameter can be any literal string type and is used by shell functions or to execute SQL queries against the database. This allows the function to accept arbitrary literal string types, as well as strings that were created from other literal strings. Type checkers like mypy will reject values that are not static arguments, protecting against SQL injection attacks.

Here’s an example of a vulnerable function annotation:

And now, here’s what a conditionally safe function annotation looks like:

In this case, static analyzers will help control behavior and prevent possible vulnerabilities in your code.

StrEnum, IntEnum, IntFlag

Changes have been applied to the enum module as well. Now, right next to Enum, IntEnum, IntFlag, and Flag, we have a new base class - StrEnum. Many people were waiting for this feature to appear because there was often a need to have str values of an enum. Here’s how it was usually done:

And here’s what the implementation of the built-in IntEnum looks like:

This a brief reminder of why it wasn’t enough to use the usual enum.Enum:

The only option is to use .value during comparisons:

Previously, comparing enum elements with other strings gave False. Therefore, self-written StrEnum(str, Enum) was often used. But now, we finally have this built-in base class that's analog to record (str, Enum)

StrEnum inherits from ReprEnum, allowing for the same result between str(FruitEnum.APPLE) and format(FruitEnum.APPLE). I’d like to also remind you that previously str(FruitEnum.APPLE) would output 'FruitEnum.APPLE'. In addition to StrEnum, IntEnum with IntFlag received the same imitation.

Self annotation

The new self annotation lets you easily annotate methods that return an instance of their class.

An example from the documentation:

A new module - tomllib

This module helps with parsing TOML (JSON) files. After PEP 518, many solutions have started implementing system requirements for Python projects in pyproject.toml as opposed to setup.py. Though this module doesn’t support writing TOML, it will allow you to load already existing ones using the load method for file objects and loads for strings, respectively.

An example for loading from a TOML file:

And here’s an example for loading from TOML string:

Asynchronous TaskGroups

The TaskGroup class was added. It is an asynchronous context manager holding a group of tasks that will wait for all of them upon exit. An example here could be any asynchronous group of tasks, such as sending HTTP requests.

Here’s an example with a time delay:

Better performance

Compared to Python 3.10, we also got a 10-60% improvement in code execution performance, or, as the benchmarks show, the new version shows an average of 1.25x speedup. Let me remind you that for the version of Python 3.11, Guido van Rossum (the author of the Python language) promised a twofold increase in speed. And though such impressive results haven’t been achieved, the CPython development team continues to work on the implementation of the interpreter to maintain its position.

But nevertheless, Python remains a leader in the TIOBE Index in April 2023, overtaking such programming languages as Java, JS, and C/C++.

Bottom line

Here’s what we got in this update when comparing it with Python 3.10:

1. A small speed increase.
2. Better tracking, which lets you save time on finding errors.
3. The addition of the long-awaited StrEnum.
4. A new except* statement and an exception group were added.
5. A new add_note() method to extend exception message.
6. Required and NotRequired fields in TypedDict, and so on.
   

Modern market dynamics require us to constantly learn new things and respond to new trends as quickly as possible. I'm sure that new functionality will update the list of best practices in the near future, meaning that you, as a developer, will be able to speed up the development and maintenance of your code.

Thanks for reading. Glory to Ukraine 🇺🇦

Sources:

What’s New In Python 3.11
PEP 654 – Exception Groups and except* PEP 518 – Specifying Minimum Build System Requirements for Python Projects