summaryrefslogtreecommitdiff
path: root/py
AgeCommit message (Collapse)Author
2022-08-26py/objstr: Optimise mp_obj_new_str_from_vstr for known-safe strings.Jim Mussared
The new `mp_obj_new_str_from_utf8_vstr` can be used when you know you already have a unicode-safe string. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26py/objstr: Always ensure mp_obj_str_from_vstr is unicode-safe.Jim Mussared
Now that we have `mp_obj_new_str_type_from_vstr` (private helper used by objstr.c) split from the public API (`mp_obj_new_str_from_vstr`), we can enforce a unicode check at the public API without incurring a performance cost on the various objstr.c methods (which are already working on known unicode-safe strings). Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-26py/objstr: Split mp_obj_str_from_vstr into bytes/str versions.Jim Mussared
Previously the desired output type was specified. Now make the type part of the function name. Because this function is used in a few places this saves code size due to smaller call-site. This makes `mp_obj_new_str_type_from_vstr` a private function of objstr.c (which is almost the only place where the output type isn't a compile-time constant). This saves ~140 bytes on PYBV11. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-23py/builtinimport: Allow overriding of mp_builtin___import__.Laurens Valk
This allows ports to override mp_builtin___import__. This can be useful in MicroPython applications where MICROPY_ENABLE_EXTERNAL_IMPORT has to be disabled due to its impact on build size (2% to 2.5% of the minimal port). By overriding the otherwise very minimal mp_builtin___import__, ports can still allow limited forms of application-specific imports. Signed-off-by: Laurens Valk <laurens@pybricks.com>
2022-08-23py: Clean up formatting of union definitions.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2022-08-19py/dynruntime: Add mp_obj_is_true.Jim Mussared
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-18all: Remove MICROPY_PY_IO_FILEIO config option.Damien George
Since commit e65d1e69e88268145ff0e7e73240f028885915be there is no longer an io.FileIO class, so this option is no longer needed. This option also controlled whether or not files supported being opened in binary mode (eg 'rb'), and could, if disabled, lead to confusion as to why opening a file in binary mode silently did the wrong thing (it would just open in text mode if MICROPY_PY_IO_FILEIO was disabled). The various VFS implementations (POSIX, FAT, LFS) were the only places where enabling this option made a difference, and in almost all cases where one of these filesystems were enabled, MICROPY_PY_IO_FILEIO was also enabled. So it makes sense to just unconditionally enable this feature (ability to open a file in binary mode) in all cases, and so just remove this config option altogether. That makes configuration simpler and means binary file support always exists (and opening a file in binary mode is arguably more fundamental than opening in text mode, so if anything should be configurable then it should be the ability to open in text mode). Signed-off-by: Damien George <damien@micropython.org>
2022-08-12py/formatfloat: Use pow(10, e) instead of pos/neg_pow lookup tables.Dan Ellis
Rework the conversion of floats to decimal strings so it aligns precisely with the conversion of strings to floats in parsenum.c. This is to avoid rendering 1eX as 9.99999eX-1 etc. This is achieved by removing the power- of-10 tables and using pow() to compute the exponent directly, and that's done efficiently by first estimating the power-of-10 exponent from the power-of-2 exponent in the floating-point representation. Code size is reduced by roughly 100 to 200 bytes by this commit. Signed-off-by: Dan Ellis <dan.ellis@gmail.com>
2022-08-12py/parsenum: Ensure that trailing zeros lead to identical results.Dan Ellis
Prior to this commit, parsenum would calculate "1e-20" as 1.0*pow(10, -20), and "1.000e-20" as 1000.0*pow(10, -23); in certain cases, this could make seemingly-identical values compare as not equal. This commit watches for trailing zeros as a special case, and ignores them when appropriate, so "1.000e-20" is also calculated as 1.0*pow(10, -20). Fixes issue #5831.
2022-08-12py/mkrules: Use abspath to find directory for mpy-cross dependency.Damien George
Otherwise if the `mpy-cross/build/` directory doesn't exist then `mpy-cross/build/..` won't work. Signed-off-by: Damien George <damien@micropython.org>
2022-08-12py/objstr: Remove str function object declarations from header file.Damien George
Since f7f56d42851aaff2027e23a8ca45c1f1973f1aca consolidated all uses of these to a single locals dict, they no longer need to be made public. Signed-off-by: Damien George <damien@micropython.org>
2022-08-12py/objstr: Add hex/fromhex to bytes/memoryview/bytearray.Jim Mussared
These were added in Python 3.5. Enabled via MICROPY_PY_BUILTINS_BYTES_HEX, and enabled by default for all ports that currently have ubinascii. Rework ubinascii to use the implementation of these methods. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-08-11py/objstr: Consolidate methods for str/bytes/bytearray/array.Andrew Leech
This commit adds the bytes methods to bytearray, matching CPython. The existing implementations of these methods for str/bytes are reused for bytearray with minor updates to match CPython return types. For details on the CPython behaviour see https://docs.python.org/3/library/stdtypes.html#bytes-and-bytearray-operations The work to merge locals tables for str/bytes/bytearray/array was done by @jimmo. Because of this merging of locals the change in code size for this commit is mostly negative: bare-arm: +0 +0.000% minimal x86: +29 +0.018% unix x64: -792 -0.128% standard[incl -448(data)] unix nanbox: -436 -0.078% nanbox[incl -448(data)] stm32: -40 -0.010% PYBV10 cc3200: -32 -0.017% esp8266: -28 -0.004% GENERIC esp32: -72 -0.005% GENERIC[incl -200(data)] mimxrt: -40 -0.011% TEENSY40 renesas-ra: -40 -0.006% RA6M2_EK nrf: -16 -0.009% pca10040 rp2: -64 -0.013% PICO samd: +148 +0.105% ADAFRUIT_ITSYBITSY_M4_EXPRESS
2022-08-11py/qstr: Change qstr hash type from mp_uint_t to size_t.Damien George
The hash is either 8 or 16 bits (depending on MICROPY_QSTR_BYTES_IN_HASH) so will fit in a size_t. This saves 268 bytes on the unix nanbox build. Non-nanbox configurations are unchanged because mp_uint_t is the same size as size_t. Signed-off-by: Damien George <damien@micropython.org>
2022-08-11py/nlrpowerpc: Fix generation of ppc64 code on ppc32 build.Efi Weiss
Due to inline assembly, wrong instructions were generated. Use corresponding 32 bit instructions and fix the offsets used. Signed-off-by: Efi Weiss <efiwiss@gmail.com>
2022-08-11py/dynruntime.mk: Allow building assembly source in natmods.Mat Booth
Allow inclusion of assembly source files in dynamic native modules.
2022-08-11all: Fix paths to mpy-cross and micropython binaries.Daniel Jour
Binaries built using the Make build system now no longer appear in the working directory of the build, but rather in the build directory. Thus some paths had to be adjusted.
2022-08-11py/mkrules.mk: Keep all build artefacts inside $(BUILD) directory.Daniel Jour
The rules for lib (static library with name $(LIBMICROPYTHON)) and the default rule to build a binary (name $(PROG)) produced outputs in the current working directory. Change this to build these files in the build directory. Note: An empty BUILD variable can cause issues (references to the root directory); this is not addressed by this commit due to multiple other places having the same issue.
2022-08-10py/objstr: Reformat str access macros to make them readable.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2022-08-10py/mpprint: Fix formatting typo with mp_print_ext_t struct name.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2022-08-06py/runtime: Fix crash in star arg unpacking.David Lechner
The reallocation trigger for unpacking star args with unknown length did not take into account the number of fixed args remaining. So it was possible that the unpacked iterators could take up exactly the memory allocated then nothing would be left for fixed args after the star args. This causes a segfault crash. This is fixed by taking into account the remaining number of fixed args in the check to decide whether to realloc yet or not. Signed-off-by: David Lechner <david@pybricks.com>
2022-08-03py/mkenv.mk: Use micropython-lib from submodule by default.Jim Mussared
Also adds micropython-lib to 'make submodules' when using a frozen manifest (for make and cmake). Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-27py/builtinimport: Remove duplicate static function argument.Angus Gratton
context==mc in all cases where this function was being called. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2022-07-26py/formatfloat: Format all whole-number floats exactly.Dan Ellis
Formerly, py/formatfloat would print whole numbers inaccurately with nonzero digits beyond the decimal place. This resulted from its strategy of successive scaling of the argument by 0.1 which cannot be exactly represented in floating point. The change in this commit avoids scaling until the value is smaller than 1, so all whole numbers print with zero fractional part. Fixes issue #4212. Signed-off-by: Dan Ellis dan.ellis@gmail.com
2022-07-26py/modio: Remove FileIO and TextIOWrapper from io module.Jim Mussared
On ports with more than one filesystem, the type will be wrong, for example if using LFS but FAT enabled, then the type will be FAT. So it's not possible to use these classes to identify a file object type. Furthermore, constructing an io.FileIO currently crashes on FAT, and make_new isn't supported on LFS. And the io.TextIOWrapper class does not match CPython at all. Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
2022-07-26py/compile: Support large integers in inline-asm data directive.Damien George
Fixes issue #8956. Signed-off-by: Damien George <damien@micropython.org>
2022-07-25py/obj: Make mp_obj_get_complex_maybe call mp_obj_get_float_maybe first.Damien George
This commit simplifies mp_obj_get_complex_maybe() by first calling mp_obj_get_float_maybe() to handle the cases corresponding to floats. Only if that fails does it attempt to extra a full complex number. This reduces code size and also means that mp_obj_get_complex_maybe() now supports user-defined classes defining __float__; in particular this allows user-defined classes to be used as arguments to cmath-module function. Furthermore, complex_make_new() can now be simplified to directly call mp_obj_get_complex(), instead of mp_obj_get_complex_maybe() followed by mp_obj_get_float(). This also improves error messages from complex with an invalid argument, it now raises "can't convert <type> to complex" rather than "can't convert <type> to float". Signed-off-by: Damien George <damien@micropython.org>
2022-07-25py/obj: Add support for __float__ and __complex__ functions.Andrew Leech
2022-07-23py/gc: Reduce code size when MICROPY_GC_SPLIT_HEAP is disabled.Rob Knegjens
Use C macros to reduce the size of firmware images when the GC split-heap feature is disabled. The code size difference of this commit versus HEAD~2 (ie the commit prior to MICROPY_GC_SPLIT_HEAP being introduced) when split-heap is disabled is: bare-arm: +0 +0.000% minimal x86: +0 +0.000% unix x64: -16 -0.003% unix nanbox: -20 -0.004% stm32: -8 -0.002% PYBV10 cc3200: +0 +0.000% esp8266: +8 +0.001% GENERIC esp32: +0 +0.000% GENERIC nrf: -20 -0.011% pca10040 rp2: +0 +0.000% PICO samd: -4 -0.003% ADAFRUIT_ITSYBITSY_M4_EXPRESS The code size difference of this commit versus HEAD~2 split-heap is enabled with MICROPY_GC_MULTIHEAP=1 (but no extra code to add more heaps): unix x64: +1032 +0.197% [incl +544(bss)] esp32: +592 +0.039% GENERIC[incl +16(data) +264(bss)]
2022-07-23py/gc: Allow the GC heap to be split over multiple memory areas.Ayke van Laethem
This commit adds a new option MICROPY_GC_SPLIT_HEAP (disabled by default) which, when enabled, allows the GC heap to be split over multiple memory areas/regions. The first area is added with gc_init() and subsequent areas can be added with gc_add(). New areas can be added at runtime. Areas are stored internally as a linked list, and calls to gc_alloc() can be satisfied from any area. This feature has the following use-cases (among others): - The ESP32 has a fragmented OS heap, so to use all (or more) of it the GC heap must be split. - Other MCUs may have disjoint RAM regions and are now able to use them all for the GC heap. - The user could explicitly increase the size of the GC heap. - Support a dynamic heap while running on an OS, adding more heap when necessary.
2022-07-18py/qstr: Make mp_decompress_rom_string decl and def the same.stijn
Fixes MSVC warning about mismatching argument types.
2022-07-18py/misc: Fix msvc compilation with compressed error messages.stijn
2022-07-18py/scheduler: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register sched_queue instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18py/modsys: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register cur_exception, sys_exitfunc, mp_sys_path_obj, mp_sys_argv_obj and sys_mutable instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18py/persistentcode: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register track_reloc_code_list instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18extmod/modbluetooth: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register `bluetooth` instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18extmod/vfs: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register vfs_cur and vfs_mount_table instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18extmod/modlwip: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register lwip_slip_stream instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18extmod/uos_dupterm: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register dupterm_objs instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18shared/runtime/pyexec: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register repl_line instead of using a conditional inside of mp_state_vm_t. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18py/mpstate: Drop MICROPY_PORT_ROOT_POINTERS from mp_state_vm_t.David Lechner
All in-tree uses of MICROPY_PORT_ROOT_POINTERS have been replaced with MP_REGISTER_ROOT_POINTER(), so now we can remove both MICROPY_PORT_ROOT_POINTERS and MICROPY_BOARD_ROOT_POINTERS from the code and remaining config files. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18shared/readline: Use MP_REGISTER_ROOT_POINTER().David Lechner
This uses MP_REGISTER_ROOT_POINTER() to register the readline_history root pointer array used by shared/readline.c and removes the registration from all mpconfigport.h files. This also required adding a new MICROPY_READLINE_HISTORY_SIZE config option since not all ports used the same sized array. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18py/make_root_pointers: Add MP_REGISTER_ROOT_POINTER parser/generator.David Lechner
This adds new compile-time infrastructure to parse source code files for `MP_REGISTER_ROOT_POINTER()` and generates a new `root_pointers.h` header file containing the collected declarations. This works the same as the existing `MP_REGISTER_MODULE()` feature. Signed-off-by: David Lechner <david@pybricks.com>
2022-07-18py/obj: Add debug-only runtime checks to mp_obj_is_type().Yonatan Goldschmidt
Zero effect on non debug builds, and also usually optimized out even in debug builds as mp_obj_is_type() is called with a compile-time known type. I'm not sure we even have dynamic uses of mp_obj_is_type() at the moment, but if we ever will they will be protected from now on. Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
2022-07-18py/obj: Add static safety checks to mp_obj_is_type().Yonatan Goldschmidt
Commit d96cfd13e3a464862c introduced a regression by breaking existing users of mp_obj_is_type(.., &mp_obj_bool). This function (and associated helpers like mp_obj_is_int()) have some specific nuances, and mistakes like this one can happen again. This commit adds mp_obj_is_exact_type() which behaves like the the old mp_obj_is_type(). The new mp_obj_is_type() has the same prototype but it attempts to statically assert that it's not called with types which should be checked using mp_obj_is_type(). If called with any of these types: int, str, bool, NoneType - it will cause a compilation error. Additional checked types (e.g function types) can be added in the future. Existing users of mp_obj_is_type() with the now "invalid" types, were translated to use mp_obj_is_exact_type(). The use of MP_STATIC_ASSERT() is not bulletproof - usually GCC (and other compilers) can't statically check conditions that are only known during link-time (like variables' addresses comparison). However, in this case, GCC is able to statically detect these conditions, probably because it's the exact same object - `&mp_type_int == &mp_type_int` is detected. Misuses of this function with runtime-chosen types (e.g: `mp_obj_type_t *x = ...; mp_obj_is_type(..., x);` won't be detected. MSC is unable to detect this, so we use MP_STATIC_ASSERT_NOT_MSC(). Compiling with this commit and without the fix for d96cfd13e3a464862c shows that it detects the problem. Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
2022-07-18py/misc: Add MP_STATIC_ASSERT_NOT_MSC().Yonatan Goldschmidt
To be used in cases where the condition of the assert does not compile under msvc. Signed-off-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
2022-07-13py/objnamedtuple: Fix segfault with empty namedtuple.Lars Haulin
The empty tuple is usually a constant object, but named tuples must be allocated to allow modification. Added explicit allocation to fix this. Also added a regression test to verify creating an empty named tuple works. Fixes issue #7870. Signed-off-by: Lars Haulin <lars.haulin@gmail.com>
2022-07-12py/vm: Consistently indent #if guards to match the code they surround.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2022-07-12py/vm: In YIELD_FROM opcode, expand helper macros and remove them.Damien George
The GENERATOR_EXIT_IF_NEEDED macro is only used once and it's easier to read and understand the code if this macro body is written in the code. Then the comment just before it makes more sense. Signed-off-by: Damien George <damien@micropython.org>
2022-07-12py/vm: Remove check for ip being NULL when handling StopIteration.Damien George
This check for code_state->ip being NULL was added in a7c02c4538bb2b986efb1999e00da4d76345767d with a commit message that "When generator raises exception, it is automatically terminated (by setting its code_state.ip to 0)". It was also added without any tests to test for this particular case. (The commit did mention that CPython's test_pep380.py triggered a bug, but upon re-running this test it did not show any need for this NULL check of code_state->ip.) It is true that generators that have completed (either by running to their end or raising an exception) set "code_state.ip = 0". But there is an explicit check at the start of mp_obj_gen_resume() to return immediately for any attempt to resume an already-stopped generator. So the VM can never execute a generator with NULL ip (and this was true at the time of the above-referenced commit). Furthermore, the other parts of the VM just before and after this piece of code do require (or at least assume) code_state->ip is non-NULL. Signed-off-by: Damien George <damien@micropython.org>