summaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)Author
2025-05-18tools/mpremote/tests: Add tests for 'fs tree' command.Jos Verlinde
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-05-18tools/mpremote: Add new 'fs tree' command.Jos Verlinde
Add `mpremote fs tree` command to show a tree of the device's files. It: - Shows a treeview from current path or specified path. - Uses the graph chars ("├── ", "└── ") (not configurable). - Has the options: -v/--verbose adds the serial device name to the top of the tree -s/--size add a size to the files -h/--human add a human readable size to the files Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-05-16esp32: Update to use ESP-IDF v5.4.1.IhorNehrutsa
This version of the IDF uses about 1KB more IRAM and 1KB more DRAM on most boards, but 6.5KB more DRAM usage on the S3. It seems that's due to a lot of small increases in many components. Signed-off-by: Ihor Nehrutsa <Ihor.Nehrutsa@gmail.com>
2025-05-16tools/gen-cpydiff.py: Ensure every item has at least 2 TOC levels.Jeff Epler
Previously, the navigation ended up messy when the (long) description of the item became used as a 2nd level header, meaning that it was placed in the navigation. Check for this when generating cpydiff so that new cases don't sneak in unnoticed. Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-16tools/gen-cpydiff.py: Fix RST heading generation.Jeff Epler
The heading character for the difference title was always "~", but items had been added which had just a single heading level. This made the generated table of contents confused about heading levels, because heading levels are not fixed in rst, but are inferred from the order they appear in the document. Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-16tools/gen-cpydiff.py: Improve stdout vs stderr interleaving.Jeff Epler
In the syntax_space cpydiff, all the warnings were shown after the other output. This is because the output always showed all of stdout first and all of stdout second. By running Python in unbuffered mode and using `stderr=STDOUT`, the two streams are interleaved in exactly the order they're printed, so the SyntaxWarnings are interleaved with the other output. By using the `encoding=` argument of Popen, the need to explicitly convert to utf-8 is avoided. The encoding of the input also becomes utf-8 in this case, which all the test cases are (well, they're all ASCII, I think). As in `run-tests.py`, setting PYTHONIOENCODING ensures the Python interpreter's input and output are in utf-8, which is not always the case, especially on Windows systems. I spot-checked the generated doc pages and they all seemed to make sense still. Signed-off-by: Jeff Epler <jepler@gmail.com>
2025-05-15tools/verifygitlog.py: Allow long co-author and sign-off names.Daniël van de Giessen
Signed-off-by: Daniël van de Giessen <daniel@dvdgiessen.nl>
2025-05-15tools/verifygitlog.py: Disallow a leading slash in commit subject line.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2025-05-12tools/verifygitlog.py: Apply stricter rules on git subject line.Damien George
There is a bit of ambiguity as to how the prefix of the git subject line should look like. Eg `py/vm: ...` vs `py/vm.c: ...` (whether the extension should be there or not). This commit makes the existing CI check of the git commit message stricter, by applying extra rules to the prefix, the bit before the : in the subject line. It now checks that the subject prefix: - doesn't start with unwanted bits: ., /, ports/ - doesn't have an extension: .c, .h, .cpp, .js, .rst or .md Full error messages are given when a rule does not pass. This helps to reduce maintainer burden by applying stricter rules, to keep the git commit history consistent. Signed-off-by: Damien George <damien@micropython.org>
2025-05-02tools/mpremote: For mip install, use hash to skip files that exist.Damien George
When using `mip install`, if a file that needs to be downloaded already exists locally, then the hash of that local file will be computed and if it matches the known hash of the remote file it will not be downloaded. Hashes in mip are guaranteed unique, so this change should never leave stale files on the filesystem. This behaviour follows that of the `mip` package in `micropython-lib`. Signed-off-by: Damien George <damien@micropython.org>
2025-05-02tools/ci.sh: Update URL for xtensa-lx106-elf-standalone.tar.gz.Damien George
The https://github.com/jepler/esp-open-sdk repository has been removed, so use the file hosted at micropython.org (it's the same file). Signed-off-by: Damien George <damien@micropython.org>
2025-04-29zephyr: Upgrade to Zephyr v4.0.0.Maureen Helm
Updates the Zephyr port build instructions. The CI is updated to use Zephyr docker image 0.27.4, SDK 0.17.0 and the latest Zephyr release tag. Tested on max32690fthr and frdm_k64f. Signed-off-by: Maureen Helm <maureen.helm@analog.com> Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
2025-04-26tools/mpremote: Prevent deletion of /remote files via rm -r.Jos Verlinde
Removes the risk of inadvertently deleting files on the host by preventing the deletion of files via `rm -r` on the `/remote` vfs mount point. Fixes issue #17147. Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-04-26tools/mpremote/tests: Add test for rm -r on /remote vfs.Jos Verlinde
Signed-off-by: Jos Verlinde <jos_verlinde@hotmail.com>
2025-04-26tools/mpremote: Fix possibly-missing EOPNOTSUPP errno name.Anson Mansfield
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-26tools/mpremote: Refactor error handling to apply generally to any errno.Anson Mansfield
This rewrites the code that previously manually emitted and caught various OSError subclasses with equivalent code that uses the errno name dictionary to do this generically, and updates the exception handler in do_filesystem to catch them in a similarly-generic fashion using os.strerror to retrieve an appropriate error message text equivalent to the current messages. Note that in the CPython environments where mpremote runs, the call to the OSError constructor already returns an instance of the corresponding mapped exception subtype. Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-26tools/mpremote/tests: Add tests for errno behavior.Anson Mansfield
Signed-off-by: Anson Mansfield <amansfield@mantaro.com>
2025-04-24tools/mpremote: Use zlib.compressobj instead of zlib.compress.Damien George
Because the `wbits` parameter was only added to `zlib.compress` in CPython 3.11. Using `zlib.compressobj` makes the code compatible with much older CPython versions. Fixes issue #17140. Signed-off-by: Damien George <damien@micropython.org>
2025-04-24top: Bump Ruff version to v0.11.6.Angus Gratton
Brings it into sync with a matching change to micropython-lib (which was much older). Includes one small automatic fix. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-04-09docs/reference/mpremote: Update docs for mpremote rm -r.Jos Verlinde
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09tools/mpremote/tests: Add tests for mpremote rm -r.Jos Verlinde
Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09tools/mpremote: Add recursive remove functionality to filesystem cmds.Jos Verlinde
mpremote now supports `mpremote rm -r`. Addresses #9802 and #16845. Signed-off-by: Jos Verlinde <Jos_Verlinde@hotmail.com>
2025-04-09github/workflows: Add Alif port to CI.iabdalkader
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2025-04-08tools/mpremote: Allow .img for ROMFS file and validate ROMFS image.iabdalkader
Currently the tool allows writing an invalid ROMFS image, with a bad header or images smaller than minimum size, and only checks the image extension. This commit allows deploying a ROMFS with either a ".img" or ".romfs" extension (in the future support may be added for other extensions that have different semantics, eg a manifest), and validates the image header before writing. Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2025-04-07tools/gen-cpydiff.py: Fail CPython diff generation if output matches.Angus Gratton
Previously this information was recorded in a "status" field of the result, but nothing ever parsed this result which led to non-differences not being removed. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-04-07tools/ci.sh: Manually install picotool for rp2 builds.Damien George
If picotool is not installed, it's fetched and built when compiling each rp2 board. And the "develop" branch of picotool is used instead of a release. Installing it manually using the "master" branch means the latest released version is used (instead of a possibly unstable development version), and also makes building each rp2 board a little faster. Signed-off-by: Damien George <damien@micropython.org>
2025-04-03tools/mpy_ld.py: Give better error for unsupported ARM absolute relocs.Jon Nordby
This is a known limitation, so better to give a clear warning than a catch-all AssertionError. Happens for example when trying to use soft-float on ARCH=armv6m Also give more details on the assertion for unknown relocations, such that one can see which symbol it affects etc, to aid in debugging. References issue #14430. Signed-off-by: Jon Nordby <jononor@gmail.com>
2025-03-27tools/mpremote: Make mip install skip /rom*/lib directories.Damien George
If a ROMFS is mounted then "/rom/lib" is usually in `sys.path` before the writable filesystem's "lib" entry. The ROMFS directory cannot be installed to, so skip it if found. Signed-off-by: Damien George <damien@micropython.org>
2025-03-17tools/ci.sh: Do not assume the Python interpreter is called "python".Alessandro Gatti
This commit removes the assumption made by the CI scripts that the system-provided python executable is simply named "python". The scripts will now look for a binary called "python3" first, and then fall back to "python" if that is not found. Whilst this is currently the case for the CI environment, there are no guarantees for this going forward. For example minimal CI environments set up by some developers, using the same base OS, have their python executable called "python3". Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-03-17tools/ci.sh: Build Xtensa natmods as part of the CI process.Alessandro Gatti
This commit expands the CI tests by checking whether the example native modules are able to be built for the Xtensa architecture. This was made possible by the changes to mpy_ld that allow symbol resolution across standard compiler-provided libraries. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-03-17tools/mpy_ld.py: Allow linking static libraries.Volodymyr Shymanskyy
This commit introduces an additional symbol resolution mechanism to the natmod linking process. This allows the build scripts to look for required symbols into selected libraries that are provided by the compiler installation (libgcc and libm at the moment). For example, using soft-float code in natmods, whilst technically possible, was not an easy process and required some additional work to pull it off. With this addition all the manual (and error-prone) operations have been automated and folded into `tools/mpy_ld.py`. Both newlib and picolibc toolchains are supported, albeit the latter may require a bit of extra configuration depending on the environment the build process runs on. Picolibc's soft-float functions aren't in libm - in fact the shipped libm is nothing but a stub - but they are inside libc. This is usually not a problem as these changes cater for that configuration quirk, but on certain compilers the include paths used to find libraries in may not be updated to take Picolibc's library directory into account. The bare metal RISC-V compiler shipped with the CI OS image (GCC 10.2.0 on Ubuntu 22.04LTS) happens to exhibit this very problem. To work around that for CI builds, the Picolibc libraries' path is hardcoded in the Makefile directives used by the linker, but this can be changed by setting the PICOLIBC_ROOT environment library when building natmods. Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Co-authored-by: Alessandro Gatti <a.gatti@frob.it>
2025-03-06tools/mpremote: Add romfs query, build and deploy commands.Damien George
These commands use the `vfs.rom_ioctl()` function to manage the ROM partitions on a device, and create and deploy ROMFS images. Signed-off-by: Damien George <damien@micropython.org>
2025-03-05tools/mpy-tool.py: Support calling main() from an external script.Volodymyr Shymanskyy
Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-03-05tools/mpy-tool.py: Add support for self-hosting of mpy-tool.Volodymyr Shymanskyy
This allows running mpy-tool using MicroPython itself. An appropriate test is added to CI to make sure it continues to work. Signed-off-by: Volodymyr Shymanskyy <vshymanskyi@gmail.com> Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-28tools/mpremote/tests: Add test for RemoteFile.readline.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2025-02-28tools/mpremote: Optimise readline support in mount.Andrew Leech
This significantly speeds up readline on files opened directly from an mpremote mount. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
2025-02-25all: Upgrade to ruff v0.9.6.Christian Clauss
Signed-off-by: Christian Clauss <cclauss@me.com>
2025-02-25github/workflows: Include the Python version in the ESP-IDF cache key.Angus Gratton
Changing runner OS can change Python version, and ESP-IDF installs are keyed on ESP-IDF and Python version together. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-25github/workflows: Stop using ubuntu-20.04.Damien George
For GitHub Actions, ubuntu-20.04 is deprecated and will be removed by 1st April 2025. See announcement at https://github.com/actions/runner-images/issues/11101 This commit changes actions that use ubuntu-20.04 to a newer image. Signed-off-by: Damien George <damien@micropython.org>
2025-02-24tools/mpremote: Add support for relative urls in package.json files.Glenn Moloney
URLs in `package.json` may now be specified relative to the base URL of the `package.json` file. Relative URLs wil work for `package.json` files installed from the web as well as local file paths. Docs: update `docs/reference/packages.rst` to add documentation for: - Installing packages from local filesystems (PR #12476); and - Using relative URLs in the `package.json` file (PR #12477); - Update the packaging example to encourage relative URLs as the default in `package.json`. Add `tools/mpremote/tests/test_mip_local_install.sh` to test the installation of a package from local files using relative URLs in the `package.json`. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
2025-02-20tools/ci.sh: Build the W5100S_EVB_PICO board with no threads.Angus Gratton
Serves as a build test for a config we don't otherwise support. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-02-12tools/pyboard.py: Make get_time use machine.RTC instead of pyb.RTC.rufusclark
The current code evaluates `pyb.RTC().datetime()` resulting in a remote side exception, as `pyb` is not defined on most ports (only stm32). The code should evaluate `machine.RTC().datetime()` and hence return the current time. Signed-off-by: rufusclark <50201718+rufusclark@users.noreply.github.com> Signed-off-by: Damien George <damien@micropython.org>
2025-02-10tools/mpremote: Support mip install from package.json on local fs.Glenn Moloney
Add support for `mpremote mip install package.json` where `package.json` is a json file on the local filesystem. Without this, package json files can only be loaded from http, https, github or gitlab URLs. This is useful for testing `package.json` files for pacages in development and for constructing one's own `package.json` files for Python packages which are not yet available for installation using mip. Signed-off-by: Glenn Moloney <glenn.moloney@gmail.com>
2025-02-10tools/ci.sh: Build MIMXRT1060_EVK with MSC enabled as part of mimxrt CI.iabdalkader
Signed-off-by: iabdalkader <i.abdalkader@gmail.com>
2025-02-10mimxrt: Add support for a UF2 bootloader.robert-hh
Allowing to use e.g. the Adafruit bootloaders with MicroPython. The .uf2 file is created in addition to the .bin and .hex files allowing to use the latter ones without the bootloader for debugging and testing. Changes: - Set the location of the ISR Vector and .text segment to 0x6000C000 and 0x6000C400. - Reserve an area at the start of ITCM for a copy of the interrupt vector table and copy the table on reset to this place. - Extend `machine.bootloader()` by setting the magic number to enable the bootloader on reset. - Create a .uf2 file which skips the segments below 0x6000C000. The bootloader has to be installed as a preparation step using the board specific methods, but then the firmware's .uf2 file version can be installed using the bootloader. The bootloader can be invoked with: - double reset - calling machine.bootloader() - Using the touch1200 method Double reset is hard to achieve on MIMXRT boards, since there is no clean reset pin. Some MIMXRT boards provide it by switching the power. Some boards are excluded from the .uf2 build: - MIMXRT1050_EVK: The uf2 bootloader is built for the QSPI version of the board. MicroPython supports the Hyperflash version. - MIMXRT1176_EVK: No support for this board yet, but it should be possible. Signed-off-by: robert-hh <robert@hammelrath.com>
2025-02-07tools/ci.sh: Add natmod tests for QEMU/Arm.Alessandro Gatti
This commit adds the natmod tests for the MPS2_AN385 board running inside QEMU to the CI pipeline. Now natmod tests capabilities are equal between the Arm and RV32 platforms for the QEMU port. Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
2025-01-29tools/mpremote: Introduce timeout_overall for read_until().Hans Maerki
And use it in `enter_raw_repl()`. This prevents waiting forever for a serial device that does not respond to the Ctrl-C/Ctrl-D/etc commands and is constantly outputting data. Signed-off-by: Hans Maerki <buhtig.hans.maerki@ergoinfo.ch>
2025-01-27tools/mpremote: Avoid initial blocking read in read_until().Hans Maerki
If the target does not return any data then `read_until()` will block indefinitely. Fix this by making the initial read part of the general read look, which always checks `inWaiting() > 0` before reading from the serial device. Also added the UART timeout to the constructor. This is not currently used but may be used as an additional safeguard. Signed-off-by: Hans Maerki <buhtig.hans.maerki@ergoinfo.ch>
2025-01-17tools/autobuild,esp32: Template the generation of esp32 port deploy.md.Angus Gratton
Allows two source files (ports/esp32/boards/deploy.md and deploy_nativeusb.md for boards with only native USB) for all esp32 installation steps, with templated chip name and flash offset inserted via string formatting. The new files add more text to explain the esptool.py port auto-detection, remove the unnecessary -z feature (already enabled by default), and add a bit of troubleshooting and port detection info. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
2025-01-16tools/pyboard.py: Wait a bit before accessing the PTY serial port.Alessandro Gatti
Some PTY targets, namely `NETDUINO2` and `MICROBIT` under Qemu, take a bit more time to present a REPL than usual. The pyboard tool is a bit too impatient and would bail out before any of those targets had a chance to respond to the raw REPL request. Co-authored-by: Damien George <damien@micropython.org> Signed-off-by: Alessandro Gatti <a.gatti@frob.it>