summaryrefslogtreecommitdiff
path: root/py
AgeCommit message (Collapse)Author
2024-03-28py/dynruntime: Add mp_binary_get_size/get_val_array/set_val_array.Damien George
These are needed to read/write array.array objects, which is useful in native code to provide fast extensions that work with big arrays of data. Signed-off-by: Damien George <damien@micropython.org>
2024-03-26py/makeqstrdata.py: Ensure that scope names get low qstr values.Jim Mussared
Originally implemented in a patch file provided by @ironss-iotec. Fixes issue #14093. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2024-03-25py/nlr: Add "memory" to asm clobbers list in nlr_jump.Damien George
Newer versions of gcc (14 and up) have more sophisticated dead-code detection, and the asm clobbers list needs to contain "memory" to inform the compiler that the asm code actually does something. Tested that adding this "memory" line does not change the generated code on ARM Thumb2, x86-64 and Xtensa targets (using gcc 13.2). Fixes issue #14115. Signed-off-by: Damien George <damien@micropython.org>
2024-03-22py/compile: Add option to allow compiling top-level await.Damien George
Enabled by MICROPY_COMPILE_ALLOW_TOP_LEVEL_AWAIT. When enabled, this means that scope such as module-level functions and REPL statements can yield. The outer C code must then handle this yielded generator. Signed-off-by: Damien George <damien@micropython.org>
2024-03-22py/parse: Zero out dangling parse tree pointer to fix potential GC leak.Angus Gratton
This fixes a bug where a random Python object may become un-garbage-collectable until an enclosing Python file (compiled on device) finishes executing. Details: The mp_parse_tree_t structure is stored on the stack in top-level functions such as parse_compile_execute() in pyexec.c (and others). Although it quickly falls out of scope in these functions, it is usually still in the current stack frame when the compiled code executes. (Compiler dependent, but usually it's one stack push per function.) This means if any Python object happens to allocate at the same address as the (freed) root parse tree chunk, it's un-garbage-collectable as there's a (dangling) pointer up the stack referencing this same address. As reported by @GitHubsSilverBullet here: https://github.com/orgs/micropython/discussions/14116#discussioncomment-8837214 This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-20py/binary: Support half-float 'e' format in struct pack/unpack.Matthias Urlichs
This commit implements the 'e' half-float format: 10-bit mantissa, 5-bit exponent. It uses native _Float16 if supported by the compiler, otherwise uses custom bitshifting encoding/decoding routines. Signed-off-by: Matthias Urlichs <matthias@urlichs.de> Signed-off-by: Damien George <damien@micropython.org>
2024-03-19py/emitnative: Implement viper unary ops positive, negative and invert.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2024-03-19py/asmxtensa: Optimise asm_xtensa_mov_reg_i32_optimised() for tiny ints.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2024-03-19py/asm: Add ASM_NOT_REG and ASM_NEG_REG macros for unary ops.Damien George
ASM_NOT_REG is optional, it can be synthesised by xor(reg, -1). ASM_NEG_REG can also be synthesised with a subtraction, but most architectures have a dedicated instruction for it. Signed-off-by: Damien George <damien@micropython.org>
2024-03-18py/objdeque: Expand implementation to be doubly-ended and support iter.Dash Peters
Add `pop()`, `appendleft()`, and `extend()` methods, support iteration and indexing, and initializing from an existing sequence. Iteration and indexing (subscription) have independent configuration flags to enable them. They are enabled by default at the same level that collections.deque is enabled (the extra features level). Also add tests for checking new behavior. Signed-off-by: Damien George <damien@micropython.org>
2024-03-15py/stream: Factor stream implementations.Damien George
So there's only one location that does the ioctl(MP_STREAM_SEEK) call. Signed-off-by: Damien George <damien@micropython.org>
2024-03-15py/stream: Add mp_stream_seek() helper function.iabdalkader
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-03-15extmod/machine_usb_device: Add support for Python USB devices.Angus Gratton
This new machine-module driver provides a "USBDevice" singleton object and a shim TinyUSB "runtime" driver that delegates the descriptors and all of the TinyUSB callbacks to Python functions. This allows writing arbitrary USB devices in pure Python. It's also possible to have a base built-in USB device implemented in C (eg CDC, or CDC+MSC) and a Python USB device added on top of that. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-15py/objstr: Add a macro to define a bytes object at compile time.Angus Gratton
This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-15extmod/modmachine: Add MICROPY_PY_MACHINE_RESET configuration option.Damien George
Disabled by default, but enabled on all boards that previously had `MICROPY_PY_MACHINE_BARE_METAL_FUNCS` enabled. Signed-off-by: Damien George <damien@micropython.org>
2024-03-15extmod/modmachine: Add MICROPY_PY_MACHINE_MEMX configuration option.Damien George
Enabled by default. Signed-off-by: Damien George <damien@micropython.org>
2024-03-15extmod/modmachine: Add MICROPY_PY_MACHINE_SIGNAL configuration option.Damien George
Enabled by default. Signed-off-by: Damien George <damien@micropython.org>
2024-03-07all: Remove the "STATIC" macro and just use "static" instead.Angus Gratton
The STATIC macro was introduced a very long time ago in commit d5df6cd44a433d6253a61cb0f987835fbc06b2de. The original reason for this was to have the option to define it to nothing so that all static functions become global functions and therefore visible to certain debug tools, so one could do function size comparison and other things. This STATIC feature is rarely (if ever) used. And with the use of LTO and heavy inline optimisation, analysing the size of individual functions when they are not static is not a good representation of the size of code when fully optimised. So the macro does not have much use and it's simpler to just remove it. Then you know exactly what it's doing. For example, newcomers don't have to learn what the STATIC macro is and why it exists. Reading the code is also less "loud" with a lowercase static. One other minor point in favour of removing it, is that it stops bugs with `STATIC inline`, which should always be `static inline`. Methodology for this commit was: 1) git ls-files | egrep '\.[ch]$' | \ xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/" 2) Do some manual cleanup in the diff by searching for the word STATIC in comments and changing those back. 3) "git-grep STATIC docs/", manually fixed those cases. 4) "rg -t python STATIC", manually fixed codegen lines that used STATIC. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2024-03-04py/emitglue: Add explicit cast of proto_fun to uint8_t pointer.Damien George
Otherwise C++ compilers may complain when this header is included in an extern "C" block. Signed-off-by: Damien George <damien@micropython.org>
2024-02-29py/modthread: Move thread state initialisation to shared function.Daniël van de Giessen
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2024-02-29py/mpstate: Don't declare mp_thread_get_state().Damien George
It's instead declared by the include of `py/mpthread.h`. Signed-off-by: Damien George <damien@micropython.org>
2024-02-29py/emitbc: Remove call to adjust Python stack by 0 entries.Damien George
This call used to be needed when there was an `emit_bc_pre()` function that needed to be called at the start of each emitted bytecode. But in 8e7745eb315cdaf7dec033891f88e091ab4e016e that function was removed and now the call to `mp_emit_bc_adjust_stack_size()` does nothing when adjusting by 0 entries, so it can be removed. Signed-off-by: Damien George <damien@micropython.org>
2024-02-26py/mpconfig: Change the default enable level for the vfs module.robert-hh
For boards with MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES and up. This gets samd21 boards working (which need the vfs module in _boot.py), B_L072Z_LRWAN1, and nrf boards with smaller MCUs that use CORE or BASIC feature levels. Signed-off-by: robert-hh <robert@hammelrath.com>
2024-02-22py/emitglue: Make mp_emit_glue_assign_native's fun_data arg a const ptr.Damien George
It will only ever be read from, and in some cases (eg on esp8266) can actually be in ROM. Signed-off-by: Damien George <damien@micropython.org>
2024-02-21py/compile: Remove TODO about name mangling.Damien George
This TODO is now covered by the tests/cpydiff/core_class_name_mangling.py test. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/builtinevex: Fix setting globals for native functions in compile().Damien George
Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/emitglue: Remove n_pos_args from DEBUG_printf.Damien George
This argument was renamed in 39bf055d23be4b0f761af115773c3db1074fc2dd. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/objfun: Inline mp_obj_code_get_name() into mp_obj_fun_get_name().Damien George
The former is static and does not need to be a separate function. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/objfun: Support __name__ on native functions and generators.Damien George
This is now easy to support, since the first machine-word of a native function tells how to find the prelude, from which the function name can be extracted in the same way as for bytecode. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/emitnative: Simplify layout and loading of native function prelude.Damien George
Now native functions and native generators have similar behaviour: the first machine-word of their code is an index to get to the prelude. This simplifies the handling of these types of functions, and also reduces the size of the emitted native machine code by no longer requiring special code at the start of the function to load a pointer to the prelude. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/objfun: Split viper fun type out to separate mp_type_fun_viper type.Damien George
Viper functions are quite different to native functions and benefit from being a separate type. For example, viper functions don't have a bytecode- style prelude, and don't support generators or default arguments. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/objfun: Make mp_obj_new_fun_native/mp_obj_new_fun_asm static-inline.Damien George
To reduce code size, since they are only used once by py/emitglue.c. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/misc: Remove m_new_obj[_var]_with_finaliser macros.Damien George
They are no longer used. The new `mp_obj_malloc_with_finaliser()` macros should be used instead, which force the setting of the `base.type` field. And there's always `m_malloc_with_finaliser()` if needed. Signed-off-by: Damien George <damien@micropython.org>
2024-02-20py/obj: Introduce mp_obj_malloc_with_finaliser to allocate and set type.Damien George
Following 709e8328d9812a2e02da6fba65a03f9a873d60a2. Using this helps to reduce code size. And it ensure that the type is always set as soon as the object is allocated, which is important for the GC to function correctly. Signed-off-by: Damien George <damien@micropython.org>
2024-02-19py/obj: Change sizeof to offsetof in mp_obj_malloc_var macro.Damien George
Following b6a977848407a4ced45d118cf926bd915cc89dfb, to properly calculate the size of the variable-length allocation. Signed-off-by: Damien George <damien@micropython.org>
2024-02-19py/makeversionhdr.py: Reinstate MICROPY_GIT_HASH in mpversion.h.Damien George
MICROPY_GIT_HASH was removed in 69e34b6b6bdf45bc1111777c46839a8b5fcb30bd but it is useful for, and used by, third-party code to tell which hash of MicroPython is used. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16py/emitglue: Include fun_data_len in mp_raw_code_t only when saving.Damien George
Reduces the size of mp_raw_code_t in the case when MICROPY_DEBUG_PRINTERS is enabled. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16tools/mpy-tool.py: Skip generating frozen mp_raw_code_t when possible.Damien George
This reduces frozen code size by using the bytecode directly as the `mp_proto_fun_t`. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16py/emitglue: Introduce mp_proto_fun_t as a more general mp_raw_code_t.Damien George
Allows bytecode itself to be used instead of an mp_raw_code_t in the simple and common cases of a bytecode function without any children. This can be used to further reduce frozen code size, and has the potential to optimise other areas like importing. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16py/emitglue: Simplify mp_raw_code_t's kind and scope_flags members.Damien George
To simplify their access and reduce code size. The `scope_flags` member is only ever used to determine if a function is a generator or not, so make it reflect that fact as a bool type. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16py/emitglue: Provide a truncated mp_raw_code_t for non-asm code.Damien George
The `asm_n_pos_args` and `asm_type_sig` members of `mp_raw_code_t` are only used for raw codes of type MP_CODE_NATIVE_ASM, which are rare, for example in frozen code. So using a truncated `mp_raw_code_t` in these cases helps to reduce frozen code size on targets that have MICROPY_EMIT_INLINE_ASM enabled. With this, change in firmware size of RPI_PICO builds is -648. Signed-off-by: Damien George <damien@micropython.org>
2024-02-16py/emitglue: Reorder and resize members of mp_raw_code_t.Damien George
The mp_raw_code_t struct has been reordered and some members resized. The `n_pos_args` member is renamed to `asm_n_pos_args`, and `type_sig` renamed to `asm_type_sig` to indicate that these are used only for the inline-asm emitters. These two members are also grouped together in the struct. The justifications for resizing the members are: - `fun_data_len` can be 32-bits without issue - `n_children` is already limited to 16-bits by `mp_emit_common_t::ct_cur_child` - `scope_flags` is already limited to 16-bits by `scope_t::scope_flags` - `prelude_offset` is already limited to 16-bits by the argument to `mp_emit_glue_assign_native()` - it's reasonable to limit the maximim number of inline-asm arguments to 12 (24 bits for `asm_type_sig` divided by 2) This change helps to reduce frozen code size (and in some cases RAM usage) in the following cases: - 64-bit targets - builds with MICROPY_PY_SYS_SETTRACE enabled - builds with MICROPY_EMIT_MACHINE_CODE enabled but MICROPY_EMIT_INLINE_ASM disabled With this change, unix 64-bit builds are -4080 bytes in size. Bare-metal ports like rp2 are unchanged (because mp_raw_code_t is still 32 bytes on those 32-bit targets). Signed-off-by: Damien George <damien@micropython.org>
2024-02-07extmod/modvfs: Add new "vfs" module with mount/umount and Vfs classes.Damien George
They have been moved from the "os" module. Signed-off-by: Damien George <damien@micropython.org>
2024-01-31py/compile: Fix potential Py-stack overflow in try-finally with return.Damien George
If a return is executed within the try block of a try-finally then the return value is stored on the top of the Python stack during the execution of the finally block. In this case the Python stack is one larger than it normally would be in the finally block. Prior to this commit, the compiler was not taking this case into account and could have a Python stack overflow if the Python stack used by the finally block was more than that used elsewhere in the function. In such a scenario the last argument of the function would be clobbered by the top-most temporary value used in the deepest Python expression/statement. This commit fixes that case by making sure enough Python stack is allocated to the function. Fixes issue #13562. Signed-off-by: Damien George <damien@micropython.org>
2024-01-30py/builtinimport: Simplify calls to stat_path().Matthias Urlichs
stat_path is only called with stringified vstr_t objects. Thus, pulling the stringification into the function replaces three function calls with one, saving a few bytes. Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
2024-01-25py/mpconfig: Disable qstr hashing at minimum feature level.Jim Mussared
This will apply to bare-arm and minimal, as well as the minimal unix variant. Change the default to MICROPY_QSTR_BYTES_IN_HASH=1 for the CORE,BASIC levels, 2 for >=EXTRA. Removes explicit setting of MICROPY_QSTR_BYTES_IN_HASH==1 in ports that don't set the feature level (because 1 is implied by the default level, CORE). Applies to cc3200, pic16bt, powerpc. Removes explicit setting for nRF (which sets feature level). Also for samd, which sets CORE for d21 and FULL for d51. This means that d21 is unchanged with MICROPY_QSTR_BYTES_IN_HASH==1, but d51 now moves from 1 to 2 (roughly adds 1kiB). The only remaining port which explicitly set bytes-in-hash is rp2 because it's high-flash (hence CORE level) but lowish-SRAM, so it's worthwhile saving the RAM for runtime qstrs. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2024-01-25py/qstr: Add support for MICROPY_QSTR_BYTES_IN_HASH=0.Jim Mussared
This disables using qstr hashes altogether, which saves RAM and flash (two bytes per interned string on a typical build) as well as code size. On PYBV11 this is worth over 3k flash. qstr comparison will now be done just by length then data. This affects qstr_find_strn although this has a negligible performance impact as, for a given comparison, the length and first character will ~usually be different anyway. String hashing (e.g. builtin `hash()` and map.c) now need to compute the hash dynamically, and for the map case this does come at a performance cost. This work was funded through GitHub Sponsors. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2024-01-24ports: Fix handling of paths containing spaces in Makefiles.Iksas
Make can't handle paths with spaces, see https://savannah.gnu.org/bugs/?712 The following workarounds exist: - When using make's built-in functions: - Use relative paths wherever possible to avoid spaces in the first place. - All spaces in paths can be escaped with backslashes; quotes don't work. - Some users use the shell to temporarily rename directories, or to create symlinks without spaces. - When using make to pass commands to the system's shell, enclose paths in quotes. While make will still interpret quoted strings with spaces as multiple words, the system's shell will correctly parse the resulting command. This commit contains the following fixes: - In ports/stm32/mboot/Makefile: Use relative paths to avoid spaces when using built-in functions. - In all other files: Use quotes to enclose paths when make is used to call shell functions. All changes have been tested with a directory containing spaces. Signed-off-by: Iksas <iksas@mailbox.org>
2024-01-17py/py.mk: Remove extra build dir created for frozen_content.iabdalkader
This was originally needed because the .c --> .o rule is: $(BUILD)/%.o: %.c and because the generated frozen_content.c is inside build-FOO, it must therefore generate build-FOO/build-FOO/frozen_content.o. But 2eda5138701d6a7d36f8d8e3700d136b3c1161b7 added a new build rule for pins.c that can also be used for frozen_content.c. Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2024-01-02all: Bump version to 1.23.0-preview.v1.23.0-previewDamien George
Signed-off-by: Damien George <damien@micropython.org>