summaryrefslogtreecommitdiff
path: root/py
AgeCommit message (Collapse)Author
2025-05-29py/asmxtensa: Emit prologue jump only when constants table is in use.Alessandro Gatti
This commit simplifies native functions' prologue code by not emitting a jump opcode that goes over the function's constants pool if the pool is empty. The original code assumed the constants pool is never empty as large 32-bits constants are commonly used, but for inline assembler functions that may not be the case. This meant that inline assembler functions may start with an unneeded jump (along with its alignment byte), using four bytes more than necessary. This commit is limited to the "xtensa" target, as "xtensawin" doesn't support inline assembler functions yet, so native functions' constant pools are almost always guaranteed to hold one or more values. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-29py/emitinlinextensa: Add the rest of LX3 opcodes to the assembler.Alessandro Gatti
This commit expands the Xtensa inline assembler to support most if not all opcodes available on the ESP8266 and LX3 Xtensa cores. This is meant as a stepping stone to add inline assembler support for the ESP32 and its LX6 core, along to windowed-specific opcodes and additional opcodes that are present only on the LX7 core (ESP32-S3 and later). New opcodes being added are covered by tests, and the provided tests were expanded to also include opcodes available in the existing implementation. Given that the ESP8266 space requirements are tighter than ESP32's, certain opcodes that won't be commonly used have been put behind a define to save some space in the general use case. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-29py/asmxtensa: Make the generated code dumper work on mpy-cross.Alessandro Gatti
This commit fixes compilation errors occurring when enabling the Xtensa code dumper inside mpy-cross. The original code was meant to dump the code from an Xtensa device itself, but for debugging the inline assembler this functionality was also needed off-line. The changes involve solving a signed/unsigned mismatch that was not much of a problem for the 8266's gcc version but made modern compilers complain, and using the printf formatter for pointers when it comes to printing code addresses. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-29py/asmxtensa: Replace printf messages with exceptions.Alessandro Gatti
This commit removes old raw printf calls happening inside certain branch opcode emitters, indicating the target label is out of range for the opcode. They have been replaced with a RuntimeError being raised in these cases, using a parameterised qstr instead. Whilst this technically breaks runtime behaviour expectations, the generated code would not have worked anyway so it's better to catch those cases early. This should be updated to always emit long jumps unless jumps are backwards and short enough, following the other ports, but that's something coming later. This is actually needed because there are test files that do not work when processed through mpy-cross and entirely converted to native code. The original implementation would still generate mostly-valid code that was bound to crash on the device, whilst this change would prevent invalid code to even be emitted in the first place. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/persistentcode: Allow a port a custom commit function and track data.Damien George
Allows both MICROPY_PERSISTENT_CODE_TRACK_FUN_DATA and MP_PLAT_COMMIT_EXEC to be enabled at the same time. Signed-off-by: Damien George <damien@micropython.org>
2025-05-21py/emitnative: Clean up int-indexed Viper load/store code.Alessandro Gatti
This commit performs some minor clean up for the code involved in Viper load/store operations when said operations have an integer index. Most platform-specific code blocks were able to generate correct opcodes even when the index is 0, but they would still fall back to the general case. The general case would still emit a shortened opcode sequence so this commit does not alter the overall behaviour, but makes it easier to extend platform-specific code whenever the full index range is going to be handled rather than a subset of indices as it is now. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/asmthumb: Generate proper sequences for large register offsets.Alessandro Gatti
This commit lets the Thumb native emitter generate a proper opcode sequence when calculating an indexed register offset for load/store operations with said offset beight both greater than 65535 and not able to be represented as a shifted 8-bit bitmask. The original code would assume the scaled index would always fit in 16 bits and silently discard upper bits of the offset. Now an optimised constant loading sequence is emitted instead, and the final offset is also stored in the correct register in all cases. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/asm: Remove unused generic ASM API opcode definitions.Alessandro Gatti
This commit removes the ASM_LOAD_REG_REG and ASM_STORE_REG_REG generic ASM API opcodes from all backends, as they are not used anymore in the native emitter framework. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/emitnative: Refactor Viper register-indexed load/stores.Alessandro Gatti
This commit cleans up the Viper code generation blocks for register-indexed load and store operations. An attempt is made to simplify the code in the common code generator code block, by moving architecture-specific code to the appropriate native generation backends whenever possible. This should make that specific bit of code in the Viper generator clearer and easier to maintain in the long term. To achieve this, six generic assembler meta-opcodes have been introduced, named `ASM_{LOAD,STORE}{8,16,32}_REG_REG_REG`. A platform-independent implementation for those operations is provided, so backends that cannot emit a shorter sequence for the requested operation or are fine with the platform-independent implementation can just not provide said meta-opcodes. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/emitnative: Improve Viper register-indexed code for Thumb.Alessandro Gatti
This commit lets the Viper code generator use optimised code sequence for register-indexed load and store operations when generating Thumb code. Register-indexed load and store operations for Thumb now can take at most two machine opcodes for halfword and word values, and just a single machine opcode for byte values. The original implementation could generate up to four opcodes in the worst case (dealing with word values). Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-21py/emitnative: Improve Viper register-indexed code for Arm.Alessandro Gatti
This commit lets the Viper code generator use optimised code sequences for register-indexed load and store operations when generating Arm code. The existing code defaulted to generic multi-operations code sequences for Arm code on most cases. Now optimised implementations are provided for register-indexed loads and stores of all data sizes, taking at most two machine opcodes for each operation. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-13py/objstr: Add support for the :_b/o/x specifier in str.format.Jeff Epler
This groups non-decimal values by fours, such as bbb_bbbb_bbbb. It also supports `{:_d}` to use underscore for decimal numbers (grouped in threes). Use of incorrect ":,b" is not diagnosed. Thanks to @dpgeorge for the suggestion to reduce code size. Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-13all: Rename the "NORETURN" macro to "MP_NORETURN".Alessandro Gatti
This commit renames the NORETURN macro, indicating to the compiler that a function does not return, into MP_NORETURN to maintain the same naming convention of other similar macros. To maintain compaitiblity with existing code NORETURN is aliased to MP_NORETURN, but it is also deprecated for MicroPython v2. This changeset was created using a similar process to decf8e6a8bb940d5829ca3296790631fcece7b21 ("all: Remove the "STATIC" macro and just use "static" instead."), with no documentation or python scripts to change to reflect the new macro name. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-05-09py/emitinlinethumb: Refactor string literal as array initializer.Angus Gratton
Avoids the new Wunterminated-string-literal when compiled with gcc 15.1. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-05-09py/mpconfig: Enable io.IOBase at core feature level.Damien George
IOBase is quite an important building block of other parts of the system, such as `mpremote mount` and running .mpy and native tests. This feature costs +244 bytes of firmware size on ARM Thumb2 architectures, which is worth the cost for the extra features it enables. The change here means that `io.IOBase` is now enabled on all nrf boards, (previously it was only nRF52840 and nRF9160) and also B_L072Z_LRWAN1 (there is no change to other ports or boards). Signed-off-by: Damien George <damien@micropython.org>
2025-05-07py/makeversionhdr.py: Change utcfromtimestamp() to fromtimestamp().Jos Verlinde
The former is deprecated. Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-05-07py/modthread: Initialize thread state nlr_top to NULL.Daniël van de Giessen
This ensures the check in MP_NLR_JUMP_HEAD works as expected and nlr_jump_fail gets called so we get a bit better error message. Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-05-02py/malloc: Add mutex for tracked allocations.Angus Gratton
Fixes thread safety issue that could cause memory corruption on ports with (MICROPY_PY_THREAD && !MICROPY_PY_THREAD_GIL) - currently only rp2 and unix have this configuration. Adds unit test for TLS sockets that exercises this code path. I wasn't able to make this fail on rp2, the race condition window is pretty narrow and may not have a direct impact on a quiet system. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-04-24py: Make struct-initializing macros compatible with C++.stijn
This requires explicitly naming and initializing all members so add that where needed and possible. For MP_DEFINE_NLR_JUMP_CALLBACK_FUNCTION_1 this would require initializing the .callback member, but that's a bit of a waste since the macro is always followed by a call to nlr_push_jump_callback() to initialize exactly that member, so rewrite the macro without initializers. Signed-off-by: stijn <stijn@ignitron.net>
2025-04-22py/mkrules.cmake: Add CMake support for compressed error messages.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2025-04-22py/dynruntime.mk: Fix use of musl's libm.a when LINK_RUNTIME=1.dubiousjim
Like PICOLIBC, MUSL also has its math functions in libc.a. There is a libm.a, but it's empty. Signed-off-by: dubiousjim <dubiousjim@gmail.com>
2025-04-21py/objstr: Fix handling of OP_MODULO with namedtuple.Yoctopuce dev
This fix handles attrtuple as well, eg. os.uname(). A test case has been added in basics/attrtuple2.py. Fixes issue #16969. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-04-21py/objrange: Match CPython range slicing.Jeff Epler
The "index fixing" behavior of get_fast_slice_indexes are not desired here; the underlying behavior of mp_obj_slice_indexes actually is. Fixes issue #17016. Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-04-21all: Bump version to 1.26.0-preview.v1.26.0-previewDamien George
Signed-off-by: Damien George <damien@micropython.org>
2025-04-16all: Bump version to 1.25.0.v1.25.0Damien George
Signed-off-by: Damien George <damien@micropython.org>
2025-04-14py/emitinlinerv32: Move include of asmrv32.h to within feature guard.Damien George
Otherwise, when compiling on 16-bit systems (where `mp_uint_t` is 16 bits wide) the compiler warns about "left shift count >= width of type", from the static inline functions that have RV32_ENCODE_TYPE_xxx macros which do a lot of bit shifting. Signed-off-by: Damien George <damien@micropython.org>
2025-03-27rp2,esp32,extmod: Implement UPDATE_SUBMODULES in CMake.Angus Gratton
Rather than having Make calling CMake to generate a list of submodules and then run a Make target (which is complex and prone to masking other errors), implement the submodule update logic in CMake itself. Internal CMake-side changes are that GIT_SUBMODULES is now a CMake list, and the trigger variable name is changed from ECHO_SUBMODULES to UPDATE_SUBMODULES. The run is otherwise 100% a normal CMake run now, so most of the other special casing can be removed. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-03-27py/dynruntime: Make malloc functions raise MemoryError on failure.Damien George
Addresses some TODOs in this file. Signed-off-by: Damien George <damien@micropython.org>
2025-03-27py/mpconfig: Enable 2-argument built-in next() at basic feature level.Damien George
This is a pretty fundamental built-in and having CPython-compatible behaviour is beneficial. The code size increase is not much, and ports/boards can still disable it if needed to save space. Addresses issue #5384. Signed-off-by: Damien George <damien@micropython.org>
2025-03-17tools/mpy_ld.py: Allow linking static libraries.Volodymyr Shymanskyy
This commit introduces an additional symbol resolution mechanism to the natmod linking process. This allows the build scripts to look for required symbols into selected libraries that are provided by the compiler installation (libgcc and libm at the moment). For example, using soft-float code in natmods, whilst technically possible, was not an easy process and required some additional work to pull it off. With this addition all the manual (and error-prone) operations have been automated and folded into `tools/mpy_ld.py`. Both newlib and picolibc toolchains are supported, albeit the latter may require a bit of extra configuration depending on the environment the build process runs on. Picolibc's soft-float functions aren't in libm - in fact the shipped libm is nothing but a stub - but they are inside libc. This is usually not a problem as these changes cater for that configuration quirk, but on certain compilers the include paths used to find libraries in may not be updated to take Picolibc's library directory into account. The bare metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu 22.04LTS) happens to exhibit this very problem. To work around that for CI builds, the Picolibc libraries' path is hardcoded in the Makefile directives used by the linker, but this can be changed by setting the PICOLIBC_ROOT environment library when building natmods. Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
2025-03-06py/runtime: Automatically mount ROMFS as part of mp_init.Damien George
This is put in `mp_init()` to make it consistent across all ports. Signed-off-by: Damien George <damien@micropython.org>
2025-03-06extmod/vfs: Add mp_vfs_mount_romfs_protected() helper.Damien George
This function will attempt to create a `VfsRom` instance and mount it at location "/rom" in the filesystem. Signed-off-by: Damien George <damien@micropython.org>
2025-03-06extmod/modvfs: Add vfs.rom_ioctl function and its ioctl constants.Damien George
This is a generic interface to allow querying and modifying the read-only memory area of a device, if it has such an area. Signed-off-by: Damien George <damien@micropython.org>
2025-03-06py/objarray: Add MP_DEFINE_MEMORYVIEW_OBJ convenience macro.Damien George
This allows defining a `memoryview` instance, either statically or on the C stack. Signed-off-by: Damien George <damien@micropython.org>
2025-03-05py/makeqstrdata.py: Implement MicroPython compatibility.Volodymyr Shymanskyy
This allows running `py/makeqstrdata.py` with MicroPython itself. Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-03-05py/modsys: Add sys.implementation._build entry.Damien George
For a given MicroPython firmware/executable it can be sometimes important to know how it was built, which variant/board configuration it came from. This commit adds a new field `sys.implementation._build` that can help identify the configuration that MicroPython was built with. For now it's either: * <VARIANT> for unix, webassembly and windows ports * <BOARD>-<VARIANT> for microcontroller ports (the variant is optional) In the future additional elements may be added to this string, separated by a hyphen. Resolves issue #16498. Signed-off-by: Damien George <damien@micropython.org>
2025-03-02py/objstr: Support tuples and start/end args in startswith and endswith.Glenn Moloney
This change allows tuples to be passed as the prefix/suffix argument to the `str.startswith()` and `str.endswith()` methods. The methods will return `True` if the string starts/ends with any of the prefixes/suffixes in the tuple. Also adds full support for the `start` and `end` arguments to both methods for compatibility with CPython. Tests have been updated for the new behaviour. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
2025-02-28py/parsenum: Reduce code footprint of mp_parse_num_float.Yoctopuce dev
The mantissa parsing code uses a floating point variable to accumulate digits. Using an `mp_float_uint_t` variable instead and casting to `mp_float_t` at the very end reduces code size. In some cases, it also improves the rounding behaviour as extra digits are taken into account by the int-to-float conversion code. An extra test case handles the special case where mantissa overflow occurs while processing deferred trailing zeros. Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
2025-02-28py/emitinlinextensa: Simplify register name lookup.Alessandro Gatti
This commit changes the Xtensa inline assembly parser to use a slightly simpler (and probably a tiny bit more efficient) way to look up register names when decoding instruction parameters. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-02-27py/emitinlinerv32: Reduce the footprint of compiled code.Alessandro Gatti
This commit introduces a few changes aimed at reducing the amount of space taken by the inline assembler once compiled: * The register string table uses 2 bytes for each qstr rather than the usual 4 * The opcode table uses 2 bytes for each qstr rather than the usual 4 * Opcode masks are not embedded in each opcode entry but looked up via an additional smaller table, reducing the number of bytes taken by an opcode's masks from 12 to 2 (with a fixed overhead of 24 bytes for the the masks themselves stored elsewhere) * Some error messages had a trailing period, now removed * Error messages have been parameterised when possible, and the overall text length is smaller. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-02-27py/emitinlinerv32: Fix compilation with ESP-IDF v5.2 and later.Alessandro Gatti
This commit fixes a compilation warning (turned error) about a potentially uninitialised variable being used. The warning can be ignored as the variable in question is always written to, but the code has been changed to silence that warning. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-02-25all: Upgrade codespell to v2.4.1.Christian Clauss
This commit upgrades from codespell==2.2.6 to the current codespell==2.4.1, adding emac to the ignore-words-list. Signed-off-by: Christian Clauss <cclauss@me.com>
2025-02-18py/mkrules.cmake: Support passing CFLAGS_EXTRA in environment variable.Angus Gratton
This works similarly to the existing support in "bare metal" make ports, with the caveat that CMake will only set this value on a clean build and will reuse the previous value otherwise. This is slightly different to the CMake built-in support for CFLAGS, as this variable is used when evaluating source files for qstr generation, etc. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-11extmod/modmarshal: Add new marshal module.Damien George
This commit implements a small subset of the CPython `marshal` module. It implements `marshal.dumps()` and `marshal.loads()`, but only supports (un)marshalling code objects at this stage. The semantics match CPython, except that the actual marshalled bytes is not compatible with CPython's marshalled bytes. The module is enabled at the everything level (only on the unix coverage build at this stage). Signed-off-by: Damien George <damien@micropython.org>
2025-02-11py/persistentcode: Add mp_raw_code_save_fun_to_bytes.Damien George
Serialises a bytecode function/generator to a valid .mpy as bytes. Signed-off-by: Damien George <damien@micropython.org>
2025-02-11py/objfun: Implement function.__code__ and function constructor.Damien George
This allows retrieving the code object of a function using `function.__code__`, and then reconstructing a function from a code object using `FunctionType(code_object)`. This feature is controlled by `MICROPY_PY_FUNCTION_ATTRS_CODE` and is enabled at the full-features level. Signed-off-by: Damien George <damien@micropython.org>
2025-02-11py/objcode: Factor code object out into its own file.Damien George
The `mp_obj_code_t` and `mp_type_code` code object was defined internally in both `py/builtinevex.c` and `py/profile.c`, with completely different implementations (the former very minimal, the latter quite complete). This commit factors these implementations into a new, separate source file, and allows the code object to have four different modes, selected at compile-time: - MICROPY_PY_BUILTINS_CODE_NONE: code object not included in the build. - MICROPY_PY_BUILTINS_CODE_MINIMUM: very simple code object that just holds a reference to the function that it represents. This level is used when MICROPY_PY_BUILTINS_COMPILE is enabled. - MICROPY_PY_BUILTINS_CODE_BASIC: simple code object that holds a reference to the proto-function and its constants. - MICROPY_PY_BUILTINS_CODE_FULL: almost complete implementation of the code object. This level is used when MICROPY_PY_SYS_SETTRACE is enabled. Signed-off-by: Damien George <damien@micropython.org>
2025-02-07py/emitnative: Load and store words just once for Viper code.Alessandro Gatti
This commit fixes two Xtensa sequences in order to terminate early when loading and storing word values via an immediate index. This was meant to be part of 55ca3fd67512555707304c6b68b836eb89f09d1c but whilst it was part of the code being tested, it didn't end up in the commit. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-02-07py/emitnative: Mark condition code tables as const.Alessandro Gatti
This commit marks as const the condition code tables used when figuring out which opcode sequence must be emitted depending on the requested comparison type. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-02-07py/mkrules.mk: Reset USER_C_MODULES when building mpy-cross dependency.Andrew Leech
When a port automatically compiles `mpy-cross`, if `USER_C_MODULES` is provided by the user on the command line then it is also applied to the `mpy-cross` build. That can lead to build errors if the path is relative and not found when building `mpy-cross`. Fix that by explicitly resetting `USER_C_MODULES` when invoking the `mpy-cross` build. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>