| Age | Commit message (Collapse) | Author |
|---|
|
So these tests can run and pass using OBJ_REPR_B.
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
|
|
That is, an object whose type defines the protocol slot.
Note that due to protocol confusion, a variant of the original crasher that
returned e.g., a machine.Pin instance could still lead to a crash (#17852).
Fixes issue #17841.
Signed-off-by: Jeff Epler <jepler@gmail.com>
Signed-off-by: Jeff Epler <jepler@unpythonic.net>
|
|
This commit changes the "viper_ptr*_store_boundary" tests to make them
fail more gracefully in low memory conditions.
The original version of the tests compiled viper code blocks on the fly
when it needed them, making them fail at runtime on some boards that do
not come with enough memory for this test. This clashes with
"run-tests.py"'s ability to look for a particular signature to mark
tests as skipped due to not enough memory.
Now compiled code blocks are generated at the beginning of the test
inside an appropriate exception handler. In case of a memory error when
pre-compiling a code block, the running test exits reporting a low
memory condition to the test runner. This allows to have clean test
runs on all platforms when it comes to viper pointer tests.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
|
|
This commit fixes the encoding of conditional branch opcodes emitted for
ARMv7-M targets, when the emitter decides to use the T3 encoding for
said operation.
Fields J1 and J2 are now present in the generated opcode word, along
with correcting some minor issues in bitmasks and shifts computation.
This fixes #17940.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
|
|
ringbuffer.size must be at least 2, and is a 16-bit quantity.
This fixes several cases including the one the fuzzer discovered, which
would lead to a fatal signal when accessing the object.
Fixes issue #17847.
Signed-off-by: Jeff Epler <jepler@gmail.com>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
So that the test can run the same on all targets when used with the native
emitter.
Signed-off-by: Damien George <damien@micropython.org>
|
|
This commit provides an implementation for viper boundary tests that can
work even without big int support.
Since it uses a fixed-size buffer to hold values to work with, this
should work on any platform as long as its integers are at least 32 bits
wide, regardless its configuration on how big integers can get.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
|
|
When running the viper boundary tests, assert that the offset stores don't
clobber the base register, which is saved and temporarily modified on some
architectures.
Signed-off-by: Chris Webb <chris@arachsys.com>
|
|
Add a new MICROPY_COMP_CONST_FLOAT feature, enabled by in mpy-cross and
when compiling with MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES. The new
feature leverages the code of MICROPY_COMP_CONST_FOLDING to support folding
of floating point constants.
If MICROPY_COMP_MODULE_CONST is defined as well, math module constants are
made available at compile time. For example:
_DEG_TO_GRADIANT = const(math.pi / 180)
_INVALID_VALUE = const(math.nan)
A few corner cases had to be handled:
- The float const folding code should not fold expressions resulting into
complex results, as the mpy parser for complex immediates has
limitations.
- The constant generation code must distinguish between -0.0 and 0.0, which
are different even if C consider them as ==.
This change removes previous limitations on the use of `const()`
expressions that would result in floating point number, so the test cases
of micropython/const_error have to be updated.
Additional test cases have been added to cover the new repr() code (from a
previous commit). A few other simple test cases have been added to handle
the use of floats in `const()` expressions, but the float folding code
itself is also tested when running general float test cases, as float
expressions often get resolved at compile-time (with this change).
Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
|
|
These tests all depend on generating arbitrarily long (>64-bit) integers.
It would be possible to have these tests work in this case I think, as the
results are always masked to shorter values. But quite fiddly. So just
rename them so they are automatically skipped if the target doesn't have
big int support.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
|
|
This commit adds a fast-path optimisation for when a BUILD_SLICE is
immediately followed by a LOAD/STORE_SUBSCR for a native type, to avoid
needing to allocate the slice on the heap.
In some cases (e.g. `a[1:3] = x`) this can result in no allocations at all.
We can't do this for instance types because the get/set/delattr
implementation may keep a reference to the slice.
Adds more tests to the basic slice tests to ensure that a stack-allocated
slice never makes it to Python, and also a heapalloc test that verifies
(when using bytecode) that assigning to a slice is no-alloc.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
|
|
The test runner expects `print("SKIP")` to be followed by
`raise SystemExit`. Otherwise it waits for 10 seconds for the target to
do a soft reset before timing out and continuing.
Signed-off-by: Damien George <damien@micropython.org>
|
|
This commit reworks the Viper pointer boundary tests in order to make
them more accurate and easier to extend.
The tests are now easier to reason about in their output, using easier
to read values, and bit thresholds are now more configurable. If a new
conditional code sequence is introduced, adding a new bit threshold is
just a matter of adding a value into a tuple at the beginning of the
relevant test file.
Load tests have also been made more accurate, with better function
templates to test register-indexed operations.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
|
|
This commit adds a series of test cases to exercise the Viper code
generator load/store emitting capabilities on certain boundary
conditions.
The new test cases check whether the emitted load/store code performs
correctly when dealing with specific memory offsets, which trigger
specific code generation sequences on different architectures.
Right now the cases are for unsigned offsets whose bitmasks span up to
5, 8, and 12 bits (respectively Arm/Thumb, Xtensa, RV32).
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
|
|
This allows retrieving the code object of a function using
`function.__code__`, and then reconstructing a function from a code object
using `FunctionType(code_object)`.
This feature is controlled by `MICROPY_PY_FUNCTION_ATTRS_CODE` and is
enabled at the full-features level.
Signed-off-by: Damien George <damien@micropython.org>
|
|
A return value of 0 from Python-level `ioctl()` means success, but if
that's returned unconditionally it means that the method supports all
ioctl calls, which is not true. Returning 0 without doing anything can
potentially lead to a crash, eg for MP_STREAM_SEEK which requires returning
a value in the passed-in struct pointer.
This commit makes it so that all `ioctl()` methods respond only to
MP_STREAM_CLOSE, ie they return -1 (indicating error) for all other ioctl
calls.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
This commit adds a new `RingIO` type which exposes the internal ring-buffer
code for general use in Python programs. It has the stream interface
making it similar to `StringIO` and `BytesIO`, except `RingIO` has a fixed
buffer size and is automatically safe when reads and writes are in
different threads or an IRQ.
This new type is enabled at the "extra features" ROM level.
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
|
|
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e911b088836f4e7c37fac2bc0f67917
- 5e3006f1172d0eabbbefeb3268dfb942ec7cf9cd
- e2ff00e81113d7a3f32f860652017644b5d68bf1
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41c2fa66a224725fdb2411846c91cf1a.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
This is required because the previous commit changed the .mpy native ABI.
Signed-off-by: Damien George <damien@micropython.org>
|
|
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
Because mpy_ld.py doesn't know the target object representation, it emits
instances of `MP_OBJ_NEW_QSTR(MP_QSTR_Foo)` as const string objects, rather
than qstrs. However this doesn't work for map keys (e.g. for a locals dict)
because the map has all_keys_are_qstrs flag is set (and also auto-complete
requires the map keys to be qstrs).
Instead, emit them as regular qstrs, and make a functioning MP_OBJ_NEW_QSTR
function available (via `native_to_obj`, also used for e.g. making
integers).
Remove the code from mpy_ld.py to emit qstrs as constant strings, but leave
behind the scaffold to emit constant objects in case we want to do use this
in the future.
Strictly this should be a .mpy sub-version bump, even though the function
table isn't changing, it does lead to a change in behavior for a new .mpy
running against old MicroPython. `mp_native_to_obj` will incorrectly return
the qstr value directly as an `mp_obj_t`, leading to unexpected results.
But given that it's broken at the moment, it seems unlikely that anyone is
relying on this, so it's not work the other downsides of a sub-version bump
(i.e. breaking pure-Python modules that use @native). The opposite case of
running an old .mpy on new MicroPython is unchanged, and remains broken in
exactly the same way.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
So it can run on targets with low memory, eg esp8266.
Also enable the viper_4args() sub-test, which is now supported.
Signed-off-by: Damien George <damien@micropython.org>
|
|
See https://black.readthedocs.io/en/stable/the_black_code_style/index.html
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
Follow up from a similar fix in 426785a19eeb12aef7383fbda4693575d8c4dddf
Fixes issue #6314.
Signed-off-by: Damien George <damien@micropython.org>
|
|
This prevents a very subtle bug caused by writing e.g. `bytearray('\xfd')`
which gives you `(0xc3, 0xbd)`.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
The intent is to allow us to make breaking changes to the native ABI (e.g.
changes to dynruntime.h) without needing the bytecode version to increment.
With this commit the two bits previously used for the feature flags (but
now unused as of .mpy version 6) encode a sub-version. A bytecode-only
.mpy file can be loaded as long as MPY_VERSION matches, but a native .mpy
(i.e. one with an arch set) must also match MPY_SUB_VERSION. This allows 3
additional updates to the native ABI per bytecode revision.
The sub-version is set to 1 because the previous commits that changed the
layout of mp_obj_type_t have changed the native ABI.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
|
|
There was a missing call to MP_F_CONVERT_NATIVE_TO_OBJ.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
Native functions can just reuse the bytecode function attribute code.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Support for architecture-specific qstr linking was removed in
d4d53e9e114d779523e382c4ea38f0398e880aae, where native code was changed to
access qstr values via qstr_table. The only remaining use for the special
qstr link table in persistentcode.c is to support native module written in
C, linked via mpy_ld.py. But native modules can also use the standard
module-level qstr_table (and obj_table) which was introduced in the .mpy
file reworking in f2040bfc7ee033e48acef9f289790f3b4e6b74e5.
This commit removes the remaining native qstr liking support in
persistentcode.c's load_raw_code function, and adds two new relocation
options for constants.qstr_table and constants.obj_table. mpy_ld.py is
updated to use these relocations options instead of the native qstr link
table.
Signed-off-by: Damien George <damien@micropython.org>
|
|
And make it so this test can run on any target.
LED and time testing has been removed from this test, that can now be
tested using: ./run-tests.py --via-mpy --emit native.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
Now that constant tuples are supported in the parser, eg (1, True, "str"),
it's a small step to allow anything that is a constant to be used with the
pattern:
from micropython import const
X = const(obj)
This commit makes the required changes to allow the following types of
constants:
from micropython import const
_INT = const(123)
_FLOAT = const(1.2)
_COMPLEX = const(3.4j)
_STR = const("str")
_BYTES = const(b"bytes")
_TUPLE = const((_INT, _STR, _BYTES))
_TUPLE2 = const((None, False, True, ..., (), _TUPLE))
Prior to this, only integers could be used in const(...).
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
To make sure there are no dangling references to the lists, and the GC can
reclaim heap memory.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Prior to this commit, even with unicode disabled .py and .mpy files could
contain unicode characters, eg by entering them directly in a string as
utf-8 encoded.
The only thing the compiler disallowed (with unicode disabled) was using
\uxxxx and \Uxxxxxxxx notation to specify a character within a string with
value >= 0x100; that would give a SyntaxError.
With this change mpy-cross will now accept \u and \U notation to insert a
character with value >= 0x100 into a string (because the -mno-unicode
option is now gone, there's no way to forbid this). The runtime will
happily work with strings with such characters, just like it already works
with strings with characters that were utf-8 encoded directly.
This change simplifies things because there are no longer any feature
flags in .mpy files, and any bytecode .mpy will now run on any target.
Signed-off-by: Damien George <damien@micropython.org>
|
|
The field was renamed to _mpy in 59c5d4161120db28bc6cbc7653f2e7fdb4a87370
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
|
To not rely on sets, which are an optional feature.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Background: .mpy files are precompiled .py files, built using mpy-cross,
that contain compiled bytecode functions (and can also contain machine
code). The benefit of using an .mpy file over a .py file is that they are
faster to import and take less memory when importing. They are also
smaller on disk.
But the real benefit of .mpy files comes when they are frozen into the
firmware. This is done by loading the .mpy file during compilation of the
firmware and turning it into a set of big C data structures (the job of
mpy-tool.py), which are then compiled and downloaded into the ROM of a
device. These C data structures can be executed in-place, ie directly from
ROM. This makes importing even faster because there is very little to do,
and also means such frozen modules take up much less RAM (because their
bytecode stays in ROM).
The downside of frozen code is that it requires recompiling and reflashing
the entire firmware. This can be a big barrier to entry, slows down
development time, and makes it harder to do OTA updates of frozen code
(because the whole firmware must be updated).
This commit attempts to solve this problem by providing a solution that
sits between loading .mpy files into RAM and freezing them into the
firmware. The .mpy file format has been reworked so that it consists of
data and bytecode which is mostly static and ready to run in-place. If
these new .mpy files are located in flash/ROM which is memory addressable,
the .mpy file can be executed (mostly) in-place.
With this approach there is still a small amount of unpacking and linking
of the .mpy file that needs to be done when it's imported, but it's still
much better than loading an .mpy from disk into RAM (although not as good
as freezing .mpy files into the firmware).
The main trick to make static .mpy files is to adjust the bytecode so any
qstrs that it references now go through a lookup table to convert from
local qstr number in the module to global qstr number in the firmware.
That means the bytecode does not need linking/rewriting of qstrs when it's
loaded. Instead only a small qstr table needs to be built (and put in RAM)
at import time. This means the bytecode itself is static/constant and can
be used directly if it's in addressable memory. Also the qstr string data
in the .mpy file, and some constant object data, can be used directly.
Note that the qstr table is global to the module (ie not per function).
In more detail, in the VM what used to be (schematically):
qst = DECODE_QSTR_VALUE;
is now (schematically):
idx = DECODE_QSTR_INDEX;
qst = qstr_table[idx];
That allows the bytecode to be fixed at compile time and not need
relinking/rewriting of the qstr values. Only qstr_table needs to be linked
when the .mpy is loaded.
Incidentally, this helps to reduce the size of bytecode because what used
to be 2-byte qstr values in the bytecode are now (mostly) 1-byte indices.
If the module uses the same qstr more than two times then the bytecode is
smaller than before.
The following changes are measured for this commit compared to the
previous (the baseline):
- average 7%-9% reduction in size of .mpy files
- frozen code size is reduced by about 5%-7%
- importing .py files uses about 5% less RAM in total
- importing .mpy files uses about 4% less RAM in total
- importing .py and .mpy files takes about the same time as before
The qstr indirection in the bytecode has only a small impact on VM
performance. For stm32 on PYBv1.0 the performance change of this commit
is:
diff of scores (higher is better)
N=100 M=100 baseline -> this-commit diff diff% (error%)
bm_chaos.py 371.07 -> 357.39 : -13.68 = -3.687% (+/-0.02%)
bm_fannkuch.py 78.72 -> 77.49 : -1.23 = -1.563% (+/-0.01%)
bm_fft.py 2591.73 -> 2539.28 : -52.45 = -2.024% (+/-0.00%)
bm_float.py 6034.93 -> 5908.30 : -126.63 = -2.098% (+/-0.01%)
bm_hexiom.py 48.96 -> 47.93 : -1.03 = -2.104% (+/-0.00%)
bm_nqueens.py 4510.63 -> 4459.94 : -50.69 = -1.124% (+/-0.00%)
bm_pidigits.py 650.28 -> 644.96 : -5.32 = -0.818% (+/-0.23%)
core_import_mpy_multi.py 564.77 -> 581.49 : +16.72 = +2.960% (+/-0.01%)
core_import_mpy_single.py 68.67 -> 67.16 : -1.51 = -2.199% (+/-0.01%)
core_qstr.py 64.16 -> 64.12 : -0.04 = -0.062% (+/-0.00%)
core_yield_from.py 362.58 -> 354.50 : -8.08 = -2.228% (+/-0.00%)
misc_aes.py 429.69 -> 405.59 : -24.10 = -5.609% (+/-0.01%)
misc_mandel.py 3485.13 -> 3416.51 : -68.62 = -1.969% (+/-0.00%)
misc_pystone.py 2496.53 -> 2405.56 : -90.97 = -3.644% (+/-0.01%)
misc_raytrace.py 381.47 -> 374.01 : -7.46 = -1.956% (+/-0.01%)
viper_call0.py 576.73 -> 572.49 : -4.24 = -0.735% (+/-0.04%)
viper_call1a.py 550.37 -> 546.21 : -4.16 = -0.756% (+/-0.09%)
viper_call1b.py 438.23 -> 435.68 : -2.55 = -0.582% (+/-0.06%)
viper_call1c.py 442.84 -> 440.04 : -2.80 = -0.632% (+/-0.08%)
viper_call2a.py 536.31 -> 532.35 : -3.96 = -0.738% (+/-0.06%)
viper_call2b.py 382.34 -> 377.07 : -5.27 = -1.378% (+/-0.03%)
And for unix on x64:
diff of scores (higher is better)
N=2000 M=2000 baseline -> this-commit diff diff% (error%)
bm_chaos.py 13594.20 -> 13073.84 : -520.36 = -3.828% (+/-5.44%)
bm_fannkuch.py 60.63 -> 59.58 : -1.05 = -1.732% (+/-3.01%)
bm_fft.py 112009.15 -> 111603.32 : -405.83 = -0.362% (+/-4.03%)
bm_float.py 246202.55 -> 247923.81 : +1721.26 = +0.699% (+/-2.79%)
bm_hexiom.py 615.65 -> 617.21 : +1.56 = +0.253% (+/-1.64%)
bm_nqueens.py 215807.95 -> 215600.96 : -206.99 = -0.096% (+/-3.52%)
bm_pidigits.py 8246.74 -> 8422.82 : +176.08 = +2.135% (+/-3.64%)
misc_aes.py 16133.00 -> 16452.74 : +319.74 = +1.982% (+/-1.50%)
misc_mandel.py 128146.69 -> 130796.43 : +2649.74 = +2.068% (+/-3.18%)
misc_pystone.py 83811.49 -> 83124.85 : -686.64 = -0.819% (+/-1.03%)
misc_raytrace.py 21688.02 -> 21385.10 : -302.92 = -1.397% (+/-3.20%)
The code size change is (firmware with a lot of frozen code benefits the
most):
bare-arm: +396 +0.697%
minimal x86: +1595 +0.979% [incl +32(data)]
unix x64: +2408 +0.470% [incl +800(data)]
unix nanbox: +1396 +0.309% [incl -96(data)]
stm32: -1256 -0.318% PYBV10
cc3200: +288 +0.157%
esp8266: -260 -0.037% GENERIC
esp32: -216 -0.014% GENERIC[incl -1072(data)]
nrf: +116 +0.067% pca10040
rp2: -664 -0.135% PICO
samd: +844 +0.607% ADAFRUIT_ITSYBITSY_M4_EXPRESS
As part of this change the .mpy file format version is bumped to version 6.
And mpy-tool.py has been improved to provide a good visualisation of the
contents of .mpy files.
In summary: this commit changes the bytecode to use qstr indirection, and
reworks the .mpy file format to be simpler and allow .mpy files to be
executed in-place. Performance is not impacted too much. Eventually it
will be possible to store such .mpy files in a linear, read-only, memory-
mappable filesystem so they can be executed from flash/ROM. This will
essentially be able to replace frozen code for most applications.
Signed-off-by: Damien George <damien@micropython.org>
|
|
Signed-off-by: Damien George <damien@micropython.org>
|
