summaryrefslogtreecommitdiff
path: root/py/compile.c
AgeCommit message (Collapse)Author
2021-05-30py/compile: Raise an error on async with/for outside an async function.Jeff Epler
A simple reproducer is: async for x in (): x Before this change, it would cause an assertion error in mpy-cross and micropython-coverage.
2021-04-27py: Add option to compile without any error messages at all.Damien George
This introduces a new option, MICROPY_ERROR_REPORTING_NONE, which completely disables all error messages. To be used in cases where MicroPython needs to fit in very limited systems. Signed-off-by: Damien George <damien@micropython.org>
2020-07-25py/compile: Don't await __aiter__ special method in async-for.Jonathan Hogg
MicroPython's original implementation of __aiter__ was correct for an earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was an async-def function. But that changed in the final version of PEP492 (in CPython 3.5.2) where the function was changed to a normal one. See https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable See also the note at the end of this subsection in the docs: https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators And for completeness the BPO: https://bugs.python.org/issue27243 To be consistent with the Python spec as it stands today (and now that PEP492 is final) this commit changes MicroPython's behaviour to match CPython: __aiter__ should return an async-iterable object, but is not itself awaitable. The relevant tests are updated to match. See #6267.
2020-06-16py/compile: Implement PEP 526, syntax for variable annotations.Damien George
This addition to the grammar was introduced in Python 3.6. It allows annotating the type of a varilable, like: x: int = 123 s: str The implementation in this commit is quite simple and just ignores the annotation (the int and str bits above). The reason to implement this is to allow Python 3.6+ code that uses this feature to compile under MicroPython without change, and for users to use type checkers. In the future viper could use this syntax as a way to give types to variables, which is currently done in a bit of an ad-hoc way, eg x = int(123). And this syntax could potentially be used in the inline assembler to define labels in an way that's easier to read.
2020-06-16py/grammar.h: Consolidate duplicate sub-rules for :test and =test.Damien George
2020-06-16py/compile: Implement PEP 572, assignment expressions with := operator.Damien George
The syntax matches CPython and the semantics are equivalent except that, unlike CPython, MicroPython allows using := to assign to comprehension iteration variables, because disallowing this would take a lot of code to check for it. The new compile-time option MICROPY_PY_ASSIGN_EXPR selects this feature and is enabled by default, following MICROPY_PY_ASYNC_AWAIT.
2020-06-16py/compile: Convert scope test to SCOPE_IS_COMP_LIKE macro.Damien George
This macro can be used elsewhere.
2020-05-09py/parse: Make mp_parse_node_extract_list return size_t instead of int.Damien George
Because this function can only return non-negative values, and having the correct return type gives more information to the caller.
2020-04-23all: Format code to add space after C++-style comment start.stijn
Note: the uncrustify configuration is explicitly set to 'add' instead of 'force' in order not to alter the comments which use extra spaces after // as a means of indenting text for clarity.
2020-04-05all: Use MP_ERROR_TEXT for all error messages.Jim Mussared
2020-04-05py/objexcept: Allow compression of exception message text.Jim Mussared
The decompression of error-strings is only done if the string is accessed via printing or via er.args. Tests are added for this feature to ensure the decompression works.
2020-04-05py: Use preprocessor to detect error reporting level (terse/detailed).Jim Mussared
Instead of compiler-level if-logic. This is necessary to know what error strings are included in the build at the preprocessor stage, so that string compression can be implemented.
2020-02-28all: Reformat C and Python source code with tools/codeformat.py.Damien George
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-06py/compile: Allow 'return' outside function in minimal builds.Petr Viktorin
A 'return' statement on module/class level is not correct Python, but nothing terribly bad happens when it's allowed. So remove the check unless MICROPY_CPYTHON_COMPAT is on. This is similar to MicroPython's treatment of 'import *' in functions (except 'return' has unsurprising behavior if it's allowed).
2019-11-21py/compile: Coalesce error message for break/continue outside loop.Petr Viktorin
To reduce code size.
2019-10-05py: Add new Xtensa-Windowed arch for native emitter.Damien George
Enabled via the configuration MICROPY_EMIT_XTENSAWIN.
2019-10-04py/compile: Disallow 'import *' outside module level.Petr Viktorin
This check follows CPython's behaviour, because 'import *' always populates the globals with the imported names, not locals. Since it's safe to do this (doesn't lead to a crash or undefined behaviour) the check is only enabled for MICROPY_CPYTHON_COMPAT. Fixes issue #5121.
2019-09-26py: Rename MP_QSTR_NULL to MP_QSTRnull to avoid intern collisions.Josh Lloyd
Fixes #5140.
2019-09-26py/compile: Use calculation instead of switch to convert token to op.Damien George
2019-08-30py: Integrate sys.settrace feature into the VM and runtime.Milan Rossa
This commit adds support for sys.settrace, allowing to install Python handlers to trace execution of Python code. The interface follows CPython as closely as possible. The feature is disabled by default and can be enabled via MICROPY_PY_SYS_SETTRACE.
2019-08-30py/compile: Improve the line numbering precision for lambdas.Damien George
Prior to this patch the line number for a lambda would be "line 1" if the body of the lambda contained only a simple expression (with no line number stored in the parse node). Now the line number is always reported correctly.
2019-08-28py: Add global default_emit_opt variable to make emit kind persistent.Damien George
mp_compile no longer takes an emit_opt argument, rather this setting is now provided by the global default_emit_opt variable. Now, when -X emit=native is passed as a command-line option, the emitter will be set for all compiled modules (included imports), not just the top-level script. In the future there could be a way to also set this variable from a script. Fixes issue #4267.
2019-08-19py/compile: Improve the line numbering precision for comprehensions.Milan Rossa
The line number for comprehensions is now always reported as the correct global location in the script, instead of just "line 1".
2019-05-06py: remove "if (0)" and "if (false)" branches.Jun Wu
Prior to this commit, building the unix port with `DEBUG=1` and `-finstrument-functions` the compilation would fail with an error like "control reaches end of non-void function". This change fixes this by removing the problematic "if (0)" branches. Not all branches affect compilation, but they are all removed for consistency.
2019-05-01py/asmthumb: Support asm_thumb code running on normal ARM processors.Damien George
With this change, @micropython.asm_thumb functions will work on standard ARM processors (that are in ARM state by default), in scripts and precompiled .mpy files. Addresses issue #4675.
2019-03-14py/compile: Check that arch is set when compiling native, viper or asm.Damien George
2019-03-14py/compile: Support multiple inline asm emitters.Damien George
2019-03-14py/compile: Add support to select the native emitter at runtime.Damien George
2019-03-08py: Add support to save native, viper and asm code to .mpy files.Damien George
This commit adds support for saving and loading .mpy files that contain native code (native, viper and inline-asm). A lot of the ground work was already done for this in the form of removing pointers from generated native code. The changes here are mainly to link in qstr values to the native code, and change the format of .mpy files to contain native code blocks (possibly mixed with bytecode). A top-level summary: - @micropython.native, @micropython.viper and @micropython.asm_thumb/ asm_xtensa are now allowed in .py files when compiling to .mpy, and they work transparently to the user. - Entire .py files can be compiled to native via mpy-cross -X emit=native and for the most part the generated .mpy files should work the same as their bytecode version. - The .mpy file format is changed to 1) specify in the header if the file contains native code and if so the architecture (eg x86, ARMV7M, Xtensa); 2) for each function block the kind of code is specified (bytecode, native, viper, asm). - When native code is loaded from a .mpy file the native code must be modified (in place) to link qstr values in, just like bytecode (see py/persistentcode.c:arch_link_qstr() function). In addition, this now defines a public, native ABI for dynamically loadable native code generated by other languages, like C.
2019-03-08py/emitnative: Consolidate where HASCONSTS is set to load-const-obj fun.Damien George
Simplifies the code and fixes handling of the Ellipsis const in native code generation (which also needs the constant table so must set this flag).
2019-03-05py: Replace POP_BLOCK and POP_EXCEPT opcodes with POP_EXCEPT_JUMP.Damien George
POP_BLOCK and POP_EXCEPT are now the same, and are always followed by a JUMP. So this optimisation reduces code size, and RAM usage of bytecode by two bytes for each try-except handler.
2019-03-05py: Fix VM crash with unwinding jump out of a finally block.Damien George
This patch fixes a bug in the VM when breaking within a try-finally. The bug has to do with executing a break within the finally block of a try-finally statement. For example: def f(): for x in (1,): print('a', x) try: raise Exception finally: print(1) break print('b', x) f() Currently in uPy the above code will print: a 1 1 1 segmentation fault (core dumped) micropython Not only is there a seg fault, but the "1" in the finally block is printed twice. This is because when the VM executes a finally block it doesn't really know if that block was executed due to a fall-through of the try (no exception raised), or because an exception is active. In particular, for nested finallys the VM has no idea which of the nested ones have active exceptions and which are just fall-throughs. So when a break (or continue) is executed it tries to unwind all of the finallys, when in fact only some may be active. It's questionable whether break (or return or continue) should be allowed within a finally block, because they implicitly swallow any active exception, but nevertheless it's allowed by CPython (although almost never used in the standard library). And uPy should at least not crash in such a case. The solution here relies on the fact that exception and finally handlers always appear in the bytecode after the try body. Note: there was a similar bug with a return in a finally block, but that was previously fixed in b735208403a54774f9fd3d966f7c1a194c41870f
2019-03-01py/compile: Add optimisation to compile OrderedDict inplace.Damien George
This optimisation eliminates the need to create a temporary normal dict. The optimisation is enabled via MICROPY_COMP_CONST_LITERAL which is enabled by default (although only has an effect if OrderdDict is enabled). Thanks to @pfalcon for the initial idea and implementation.
2019-02-26py/compile: Fix handling of unwinding BaseException in async with.Damien George
All exceptions that unwind through the async-with must be caught and BaseException is the top-level class, which includes Exception and others. Fixes issue #4552.
2019-02-25py: Eliminate warnings about unused arguments when debugging disabled.Damien George
2019-02-12py: Downcase all MP_OBJ_IS_xxx macros to make a more consistent C API.Damien George
These macros could in principle be (inline) functions so it makes sense to have them lower case, to match the other C API functions. The remaining macros that are upper case are: - MP_OBJ_TO_PTR, MP_OBJ_FROM_PTR - MP_OBJ_NEW_SMALL_INT, MP_OBJ_SMALL_INT_VALUE - MP_OBJ_NEW_QSTR, MP_OBJ_QSTR_VALUE - MP_OBJ_FUN_MAKE_SIG - MP_DECLARE_CONST_xxx - MP_DEFINE_CONST_xxx These must remain macros because they are used when defining const data (at least, MP_OBJ_NEW_SMALL_INT is so it makes sense to have MP_OBJ_SMALL_INT_VALUE also a macro). For those macros that have been made lower case, compatibility macros are provided for the old names so that users do not need to change their code immediately.
2019-01-27py/compile: Swap order of pop_block/pop_except in "except as" handler.Damien George
To make the try-finally block self contained.
2018-10-28py/scope: Optimise scope_find_or_add_id to not need "added" arg.Damien George
Taking the address of a local variable is mildly expensive, in code size and stack usage. So optimise scope_find_or_add_id() to not need to take a pointer to the "added" variable, and instead take the kind to use for newly added identifiers.
2018-10-28py/compile: Remove unneeded variable from global/nonlocal stmt helpers.Damien George
2018-10-28py/compile: Fix case of eager implicit conversion of local to nonlocal.Damien George
This ensures that implicit variables are only converted to implicit closed-over variables (nonlocals) at the very end of the function scope. If variables are closed-over when first used (read from, as was done prior to this commit) then this can be incorrect because the variable may be assigned to later on in the function which means they are just a plain local, not closed over. Fixes issue #4272.
2018-10-01py/emitnative: Implement yield and yield-from in native emitter.Damien George
This commit adds first class support for yield and yield-from in the native emitter, including send and throw support, and yields enclosed in exception handlers (which requires pulling down the NLR stack before yielding, then rebuilding it when resuming). This has been fully tested and is working on unix x86 and x86-64, and stm32. Also basic tests have been done with the esp8266 port. Performance of existing native code is unchanged.
2018-09-27py/emitnative: Place const objs for native code in separate const table.Damien George
This commit changes native code to handle constant objects like bytecode: instead of storing the pointers inside the native code they are now stored in a separate constant table (such pointers include objects like bignum, bytes, and raw code for nested functions). This removes the need for the GC to scan native code for root pointers, and takes a step towards making native code independent of the runtime (eg so it can be compiled offline by mpy-cross). Note that the changes to the struct scope_t did not increase its size: on a 32-bit architecture it is still 48 bytes, and on a 64-bit architecture it decreased from 80 to 72 bytes.
2018-09-20py: Shorten error messages by using contractions and some rewording.Damien George
2018-09-15py/emitnative: Make viper funcs run with their correct globals context.Damien George
Viper functions will now capture the globals at the point they were defined and use these globals when executing.
2018-09-15py: Make viper functions have the same entry signature as native.Damien George
This commit makes viper functions have the same signature as native functions, at the level of the emitter/assembler. This means that viper functions can now be wrapped in the same uPy object as native functions. Viper functions are now responsible for parsing their arguments (before it was done by the runtime), and this makes calling them more efficient (in most cases) because the viper entry code can be custom generated to suit the signature of the function. This change also opens the way forward for viper functions to take arbitrary numbers of arguments, and for them to handle globals correctly, among other things.
2018-09-15py/compile: Factor code that compiles viper type annotations.Damien George
2018-09-15py/compile: Merge viper annotation and normal param compilation stages.Damien George
Now that the compiler can store the results of the viper types in the scope, the viper parameter annotation compilation stage can be merged with the normal parameter compilation stage.
2018-09-15py/emit: Completely remove set_native_type, arg type is set in compiler.Damien George
In viper mode, the type of the argument is now stored in id_info->flags.
2018-09-15py/emit: Remove need to call set_native_type to set viper return type.Damien George
Instead this return type is now stored in the scope_flags.
2018-09-15py/emit: Remove need to call set_native_type to set native/viper mode.Damien George
The native emitter can easily determine the mode via scope->emit_options.