py/persistentcode: Add architecture flags compatibility checks.

This commit extends the MPY file format in a backwards-compatible way to
store an encoded form of architecture-specific flags that have been
specified in the "mpy-cross" command line, or that have been explicitly
set as part of a native emitter configuration.

The file format changes are as follows:

* The features byte, previously containing the target native
  architecture and the minor file format version, now claims bit 6 as a
  flag indicating the presence of an encoded architecture flags integer
* If architecture flags need to be stored, they are placed right after
  the MPY file header.

This means that properly-written MPY parsers, if encountering a MPY file
containing encoded architecture flags, should raise an error since no
architecture identifiers have been defined that make use of bits 6 and
7 in the referenced header byte.  This should give enough guarantees of
backwards compatibility when this feature is used (improper parsers were
subjected to breakage anyway).

The encoded architecture flags could have been placed at the end, but:

* Having them right after the header makes the architecture
  compatibility checks occur before having read the whole file in memory
  (which still happens on certain platforms as the reader may be backed
  by a memory buffer), and prevents eventual memory allocations that do
  not take place if the module is rejected early
* Properly-written MPY file parsers should have checked the upper two
  bits of the flags byte to be actually zero according to the format
  specification available right before this change, so no assumptions
  should have been made on the exact order of the chunks for an
  unexpected format.

The meaning of the architecture flags value is backend-specific, with
the only common characteristic of being a variable-encoded unsigned
integer for the time being.

The changes made to the file format effectively limit the number of
possible target architectures to 16, of which 13 are already claimed.
There aren't that many new architectures planned to be supported for the
lifetime of the current MPY file format, so this change still leaves
space for architecture updates if needed.

Signed-off-by: Alessandro Gatti <a.gatti@frob.it>

1 files changed, 26 insertions, 1 deletions

diff --git a/docs/reference/mpyfiles.rst b/docs/reference/mpyfiles.rst
index cd4db623f..b2d552d09 100644
--- a/docs/reference/mpyfiles.rst
+++ b/docs/reference/mpyfiles.rst
@@ -80,6 +80,10 @@ If importing an .mpy file fails then try the following:
   above, or by inspecting the ``MPY_CROSS_FLAGS`` Makefile variable for the
   port that you are using.
 
+* If the third byte of the .mpy file has bit #6 set, then check whether the
+  encoded architecture-specific flag bits vuint is compatible with the
+  target you're importing the file on.
+
 The following table shows the correspondence between MicroPython release
 and .mpy version.
 
@@ -153,10 +157,31 @@ size    field
 ======  ================================
 byte    value 0x4d (ASCII 'M')
 byte    .mpy major version number
-byte    native arch and minor version number (was feature flags in older versions)
+byte    feature flags, native arch, minor version number (was feature flags in older versions)
 byte    number of bits in a small int
 ======  ================================
 
+The third byte is split as follows (MSB first):
+
+======  ================================
+bit     meaning
+======  ================================
+7       reserved, must be 0
+6       an architecture-specific flags vuint follows the header
+5..2    native arch number
+1..0    minor version number
+======  ================================
+
+Architecture-specific flags
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If bit #6 of the header's feature flags byte is set, then a vuint containing
+optional architecture-specific information will follow the header. The contents
+of this integer depends on which native architecture the file is meant for.
+
+See also ``mpy-tool.py``'s ``-march-flags`` command-line option to set this
+value when creating MPY files.
+
 The global qstr and constant tables
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~