summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2020-12-07stm32/Makefile: Change -O0 to -Og for DEBUG=1 builds.Damien George
The -Og optimisation level produces a more realistic build, gives a better debugging experience, and generates smaller code than -O0, allowing debug builds to fit in flash. This commit also assigns variables in can.c to prevent warnings when -Og is used, and builds a board in CI with DEBUG=1 enabled. Signed-off-by: Damien George <damien@micropython.org>
2020-12-01tools/pyboard.py: Add fast raw-paste mode.Damien George
This commit adds support to pyboard.py for the new raw REPL paste mode. Note that this new pyboard.py is fully backwards compatible with old devices (it detects if the device supports the new raw REPL paste mode). Signed-off-by: Damien George <damien@micropython.org>
2020-11-29tools/ci.sh: Add helper script to run CI tasks.Damien George
The aim is for this script to be used on any CI platform, as well as run locally. Signed-off-by: Damien George <damien@micropython.org>
2020-10-20tools/upip.py: Support explicit port number in host.awachtler
Adding a port number other then 443 to a PyPI URL may be needed if a local server like devpi is used.
2020-09-11tools: Write msvc-compatible frozen content.stijn
The msvc compiler doesn't accept zero-sized arrays so let the freezing process generate compatible C code in case no modules are found and the involved arrays are all empty. This doesn't affect the functionality in any way because those arrays only get accessed when mp_frozen_mpy_names contains names, i.e. when modules are actually found.
2020-09-09tools/mpy-tool.py: Fix merge of multiple mpy files to POP_TOP correctly.Damien George
MP_BC_CALL_FUNCTION will leave the result on the Python stack, so that result must be discarded by MP_BC_POP_TOP. Signed-off-by: Damien George <damien@micropython.org>
2020-08-29all: Update Python code to conform to latest black formatting.Damien George
Updating to Black v20.8b1 there are two changes that affect the code in this repository: - If there is a trailing comma in a list (eg [], () or function call) then that list is now written out with one line per element. So remove such trailing commas where the list should stay on one line. - Spaces at the start of """ doc strings are removed. Signed-off-by: Damien George <damien@micropython.org>
2020-08-22tools/makemanifest.py: Use os.makedirs to make path for generated files.Damien George
The existing implementation of mkdir() in this file is not sophisticated enough to work correctly on all operating systems (eg Mac can raise EISDIR). Using the standard os.makedirs() function handles all cases correctly. Signed-off-by: Damien George <damien@micropython.org>
2020-08-21tools/mpy-tool.py: Fix offset of line number info.Martin Milata
Signed-off-by: Martin Milata <martin@martinmilata.cz>
2020-08-21tools/pyboard.py: Replace eval() of received data with alternative.Michael Buesch
Prior to this commit, pyboard.py used eval() to "parse" file data received from the board. Using eval() on received data from a device is dangerous, because a malicious device may inject arbitrary code execution on the PC that is doing the operation. Consider the following scenario: Eve may write a malicious script to Bob's board in his absence. On return Bob notices that something is wrong with the board, because it doesn't work as expected anymore. He wants to read out boot.py (or any other file) to see what is wrong. What he gets is a remote code execution on his PC. Proof of concept: Eve: $ cat boot.py _print = print print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04") $ ./pyboard.py -f cp boot.py : cp boot.py :boot.py Bob: $ ./pyboard.py -f cp :boot.py /tmp/foo cp :boot.py /tmp/foo bin chroot dev home lib32 media opt root sbin sys usr boot config etc lib lib64 mnt proc run srv tmp var Pwned! There's also the possibility that the device is malfunctioning and sends random and possibly dangerous data back to the PC, to be eval'd. Fix this problem by using ast.literal_eval() to parse the received bytes, instead of eval(). Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-08tools/makemanifest.py: Print nicely formatted errors from mpy-cross.Damien George
If mpy-cross exits with an error be sure to print that error in a way that is readable, instead of a long bytes object. Signed-off-by: Damien George <damien@micropython.org>
2020-07-26tools/makemanifest.py: Use errno.EEXIST instead of number 17.Damien George
To make this code more portable, across different platforms. Signed-off-by: Damien George <damien@micropython.org>
2020-07-20lib/mbedtls_errors: Add code to patch mbedtls for shortened error strs.Thorsten von Eicken
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of `mbedtls/library/error.c` to give shorter error strings, reducing the build size of the error strings from about 12-16kB down to about 2-5kB.
2020-07-01Revert "tools/pydfu.py: Respect longer timeouts requested by DFU dev..."Andrew Leech
This reverts commit 4d6f60d4286b3e6ea8c9fb0141ba13e4978882b6. This implementation used the timeout as a maximum amount of time needed for the operation, when actually the spec and other tools suggest that it's the minumum delay needed between subsequent USB transfers.
2020-06-30tools/pydfu.py: Respect longer timeouts requested by DFU device/mboot.Andrew Leech
2020-06-24tools/makemanifest.py: Support freezing a subdirectory recursively.Damien George
This adds support for freezing an entire directory while keeping the directory as part of the import path. For example freeze("path/to/library", "module") will recursively freeze all scripts in "path/to/library/module" and have them importable as "from module import ...". Signed-off-by: Damien George <damien@micropython.org>
2020-06-19tools/uncrustify: Enable more opts to remove space between func and '('.David Lechner
With only `sp_func_proto_paren = remove` set there are some cases where uncrustify misses removing a space between the function name and the opening '('. This sets all of the related options to `force` as well.
2020-06-18tools/codeformat.py: Include extmod/{btstack,nimble} in code formatting.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2020-06-14tools/uncrustify: Update config for v0.71.0.David Lechner
This is the result of running... uncrustify -c tools/uncrustify.cfg --update-config-with-doc -o tools/uncrustify.cfg ...with some manual fixups to correct places where it changed things it should not have. Essentially it just adds new config parameters introduced in v0.71.0 with their default values. Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14tools/codeformat.py: Remove sizeof fixup.David Lechner
Formatting for `* sizeof` was fixed in uncrustify v0.71, so we no longer need the fixups for it. Also, there was one file where the updated uncrustify caught a problem that the regex didn't pick up, which is updated in this commit. Signed-off-by: David Lechner <david@pybricks.com>
2020-06-14tools/uncrustify.cfg: Remove deprecated sp_word_brace option.David Lechner
This option was removed in uncrustify v0.71. Signed-off-by: David Lechner <david@pybricks.com>
2020-05-29tools/codeformat.py: Add verbose option to pass to uncrustify and black.David Lechner
This adds a new command line option `-v` to `tools/codeformat.py` to enable verbose printing of all files that are scanned. Normally `uncrustify` and `black` are called with the `-q` option so setting verbose suppresses the `-q` option and on `black` also enables the `-v` option which makes it print out all file names matching the filter similar to how `uncrustify` does by default.
2020-05-28tools/codeformat.py: Use -q option on uncrustify to make output quiet.David Lechner
This suppresses the Parsing: <file> as language C lines. This makes parsing run a bit faster and on CI it makes for less scrolling through logs (and black already uses the -q option).
2020-04-23all: Format code to add space after C++-style comment start.stijn
Note: the uncrustify configuration is explicitly set to 'add' instead of 'force' in order not to alter the comments which use extra spaces after // as a means of indenting text for clarity.
2020-04-22tools/check_code_size.sh: Remove unused script.Damien George
It's now replaced by tools/metrics.py.
2020-04-22tools/metrics.py: Use OrderedDict when reading build log.Damien George
So that the output (eg of the diff command) always has the lines in the same order.
2020-04-22tools/metrics.py: Don't build mpy-cross if not needed by any ports.Damien George
To save build time.
2020-04-22tools/metrics.py: Add option to diff to error if delta above threshold.Damien George
Useful for things like CI where the size check is automated.
2020-04-18all: Fix implicit conversion from double to float.stijn
These are found when building with -Wfloat-conversion.
2020-04-05py/objexcept: Allow compression of exception message text.Jim Mussared
The decompression of error-strings is only done if the string is accessed via printing or via er.args. Tests are added for this feature to ensure the decompression works.
2020-03-30tests: Format all Python code with black, except tests in basics subdir.David Lechner
This adds the Python files in the tests/ directory to be formatted with ./tools/codeformat.py. The basics/ subdirectory is excluded for now so we aren't changing too much at once. In a few places `# fmt: off`/`# fmt: on` was used where the code had special formatting for readability or where the test was actually testing the specific formatting.
2020-03-30tools/pyboard.py: Add -d as an alias for --device.Lars Kellogg-Stedman
2020-03-30tools/pyboard.py: Support setting device/baudrate from shell env vars.Lars Kellogg-Stedman
Allow defaults for --device and --baudrate to be set in the environment using PYBOARD_DEVICE and PYBOARD_BAUDRATE.
2020-03-28all: Remove spaces inside and around parenthesis.Damien George
Using new options enabled in the uncrustify configuration.
2020-03-25all: Remove spaces between nested paren and inside function arg paren.Damien George
Using new options enabled in the uncrustify configuration.
2020-03-25tools/codeformat.py: Include all msvc C code in auto-format.stijn
2020-03-22tools/pydfu.py: Display any error strings from device/mboot.Andrew Leech
2020-03-22tools/pydfu.py: Add args for VID/PID & exit with cleaner error handling.Andrew Leech
2020-03-11tools/codeformat.py: Eliminate need for sizeof fixup.David Lechner
This eliminates the need for the sizeof regex fixup by rearranging things a bit. All other bitfields already use the parentheses around expressions with sizeof, so one case is fixed by following this convention. VM_MAX_STATE_ON_STACK is the only remaining problem and it can be worked around by changing the order of the operands.
2020-03-11tools/metrics.py: Use check_call instead of run to error out on error.Damien George
2020-03-05tools/makemanifest.py: Fix build on Windows by adding .exe to mpy-cross.MikeTeachman
When using a manifest on Windows the reference to mpy-cross compiler was missing the .exe file extension, so add it when appropriate. Also allow the default path to mpy-cross to be overridden by the (optional) MICROPY_MPYCROSS environment variable, to allow full flexibility on any OS.
2020-02-28all: Reformat C and Python source code with tools/codeformat.py.Damien George
This is run with uncrustify 0.70.1, and black 19.10b0.
2020-02-28tools/codeformat.py: Add formatter using uncrustify for C, black for Py.Damien George
This commit adds a tool, codeformat.py, which will reformat C and Python code to fit a certain style. By default the tool will reformat (almost) all the original (ie not 3rd-party) .c, .h and .py files in this repository. Passing filenames on the command-line to codeformat.py will reformat only those. Reformatting is done in-place. uncrustify is used for C reformatting, which is available for many platforms and can be easily built from source, see https://github.com/uncrustify/uncrustify. The configuration for uncrustify is also added in this commit and values are chosen to best match the existing code style. A small post-processing stage on .c and .h files is done by codeformat.py (after running uncrustify) to fix up some minor items: - space inserted after * when used as multiplication with sizeof - #if/ifdef/ifndef/elif/else/endif are dedented by one level when they are configuring if-blocks and case-blocks. For Python code, the formatter used is black, which can be pip-installed; see https://github.com/psf/black. The defaults are used, except for line- length which is set at 99 characters to match the "about 100" line-length limit used in C code. The formatting tools used and their configuration were chosen to strike a balance between keeping existing style and not changing too many lines of code, and enforcing a relatively strict style (especially for Python code). This should help to keep the code consistent across everything, and reduce cognitive load when writing new code to match the style.
2020-02-20tools/makemanifest.py: Support freezing with empty list of mpy files.Damien George
Fixes issue #5655.
2020-02-01tools/pyboard.py: Change shebang to use python3.Michael Buesch
This script still works with Python 2 but Python 3 is recommended.
2020-02-01tools/pyboard.py: Use slice del instead of list.clear() for Py2 compat.Michael Buesch
Python 2 does not have list.clear().
2020-02-01tools/pyboard.py: Add option --no-follow to detach after sending script.Michael Buesch
This option makes pyboard.py exit as soon as the script/command is successfully sent to the device, ie it does not wait for any output. This can help to avoid hangs if the board is being rebooted with --comman (for example). Example usage: $ python3 ./tools/pyboard.py --device /dev/ttyUSB0 --no-follow \ --command 'import machine; machine.reset()'
2020-01-23tools/pydfu.py: Clean up syntax, update comments and docstrings.c0rejump
Some parts of code have been aligned to increase readability. In general '' instead of "" were used wherever possible to keep the same convention for entire file. Import inspect line has been moved to the top according to hints reported by pep8 tools. A few extra spaces were removed, a few missing spaces were added. Comments have been updated, mostly in "read_dfu_file" function. Some other comments have been capitalized and/or slightly updated. A few docstrings were fixed as well. No real code changes intended.
2020-01-23tools: Add metrics.py script to build and compute port sizes/metrics.Damien George
2020-01-06tools/gen-cpydiff.py: Adjust subsections to sentence case.Jason Neal