teensy rips, one hundred, and link to post

README.md

-## SPI 
+# Low Level Link Speed Tests
 
-- actually-realized data rates are often smaller in practice than according to underlying specs, 
-  - see even i.e. transfer of some tens-of-megabytes onto an SD card (copying a gcode file) - we observe only ~ 20mb/sec of advertized 80mb/sec or so 
-- otherwise, you have notes on what-the-point is in your `scratch` 
-- and consider linking to the ring-test page 
-- and discuss that at the embedded limit, we often run into interrupt-request-handling-time troubles before anything else... 
+For a more conversational write-up, see [the blog post.](https://ekswhyzee.com/2024/01/04/link-layer-speed-tests.html)
 
-## TODO
+TL:DR; we want to find out actually how-fast links are between embedded systems and high-level systems, mostly for machine building, but useful across the board. In quick tests, I found that most links under-perform their apparent rates by more than half. 
 
-- test the D51 for device-to-device uart 
-- likewise for RPI (five?) to device SPI link 
+### USB
+
+![histo](usb_serial/images/2023-12-27_simple-01-baseline.png)
+> RP2040, 0.45Mbit/s
+
+![histo](usb_serial/images/2024-01-04_teensy-usb-32.png)
+> Teensy 4.0, 20Mbit/s
+
+### WizNet Ethernet 
+
+![histo](wiznet_ethernet/images/2023-12-28_eth-oneway-64.png)
+
+### SPI to PI
+
+![histo](rpi_spi/images/2024-01-04_spi-histo-10Mb.png)
 
 ---
 
-## Next
-
-- the SPI can work, I suspect
-    - probably needs that sweet sweet 250MHz-es 
-    - can crank up to 16-bit words in the FIFO, for 2x less starvation 
-        - can PIO do 32-bit words, perchance ? 
-    - fire the interrupt on transmit buffer not-full, rather than half-full ?
-        - actually no interrupt exists for this 
-    - can we do a proper echo test, then check for drops, etc, for max speed ? 
-- do the RP2040-to-RP2040 UART PIO test 
-- finish the thought, blog it up (1 day, max) 
-- take i.e. [notes](https://abyz.me.uk/rpi/pigpio/pigs.html), right ? 
-    - incl. the maybe-likely-change to use PIO spi-slave, not this bs 
-- i.e. go-notes/ref/rp2040 ? 
-    - https://cec-code-lab.aps.edu/robotics/resources/pico-c-api/index.html 
-    - https://github.com/raspberrypi/pico-examples/tree/master 
-    - https://www.circuitstate.com/tutorials/making-two-raspberry-pi-pico-boards-communicate-through-spi-using-c-cpp-sdk/ 
\ No newline at end of file
+# TODO Here 
+
+I am electing microcontrollers for the position of routers-et-cetera, and the RP2040 was a top candidate but has me spooked after these tests. I want to do some more testing:
+
+## Better Tests
+
+I want to use all of these links in the context of my link-agnostic transport layer, which currently uses shoddy ad-hoc links. I would like to have some generic wrapper code that would bundle questionable links (like these) and do i.e. error catching and test-suite-implementation in a more regular manner. It could presumably be built with a kind of simple HAL to put and to get packets.
+
+## Testing other Systems 
+
+- test the D51 for device-to-device uart 
+- the D51 and RPi 5 for big-cpu-to-small-mcu SPI test redux 
+- consider the Teensy for the router, that 600MHz and advertised-to-be-small ISR seems juicy 
+- Teensy USB might be blazing fast to boot, Paul seems like the Real Deal (TM) and Teensy 4.0 page mentions 480Mbit/s USB... 
+
+## Improving Existing Tests 
+
+- the RP2040's PIO *aught* to be killer at this, so it is likely that I simply haven't really cracked it. for example, each has an 8-space wide FIFO, each of which is 32 bits long, this means one interrupt should be able to switch a 32-byte long packet (in SPI-mode) - UART (it looks like) can go to 16 bit words 
+- we also haven't really implemented interrupts on the 2040 properly, far as I can tell: we should be able to get one latched whenever a fifo is *NOT FULL* rather than being half-full, etc, which would help prevent some chunking (earlier warning) 
+ 
\ No newline at end of file

usb_serial/code/serial_sink/plot_stamps.py

+import pandas as pd 
+import matplotlib.pyplot as plt 
+
+def plot_stamps(stamps, stamp_count, pck_len, misses):
+  # make df from stamps 
+  df = pd.DataFrame({'timestamps': stamps})
+
+  # calculate deltas between stamps 
+  df['deltas'] = df['timestamps'].diff() 
+
+  # clean NaN's 
+  df = df.dropna()
+
+  # wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us) 
+  df = df[df['deltas'] < 10000]
+
+  # Plotting
+  fig, ax1 = plt.subplots(figsize=(11, 3))
+
+  # ax1.set_xlim([0, 100])
+
+  # Primary x-axis (time deltas)
+  df['deltas'].plot(kind='hist', bins=10, ax=ax1)
+  ax1.set_xlabel('Time-Stamp Deltas (us) and equivalent (MBits/s)')
+  ax1.set_ylabel(f'Frequency (of {stamp_count})')
+
+  # get axis ticks to calculate equivalent bandwidths 
+  x_ticks = ax1.get_xticks()
+  # x_ticks = [100, 200, 300, 400, 500, 600, 700, 800, 900]
+  # ax1.set_xticks(x_ticks)
+  bandwidths = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks]
+  ticks = [] 
+
+  for i in range(len(x_ticks)):
+    print(i, x_ticks[i], bandwidths[i]) 
+    ticks.append(f"{x_ticks[i]:.0f} ({bandwidths[i]:.3f})")
+
+  ax1.set_xticklabels(ticks)
+
+  plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}, miss={misses}')
+
+  plt.tight_layout()
+
+  plt.show()
\ No newline at end of file

usb_serial/code/serial_sink/sink.py

 from cobs_usb_serial import CobsUsbSerial 
+from plot_stamps import plot_stamps
 import struct 
 import numpy as np 
-import pandas as pd 
-import matplotlib.pyplot as plt 
 
-ser = CobsUsbSerial("COM23") 
+ser = CobsUsbSerial("COM43") 
 
 stamp_count = 1000
-pck_len = 250
+pck_len = 128
 
 stamps = np.zeros(stamp_count)
 
@@ -19,41 +18,4 @@ for i in range(stamp_count):
 
 print("stamps, ", stamps)
 
-df = pd.DataFrame({'timestamps': stamps})
-
-df['deltas'] = df['timestamps'].diff() 
-
-# clean NaN's 
-df = df.dropna()
-
-# wipe obviously-wrong deltas (i.e. the 1st, which goes 0-start-us) 
-df = df[df['deltas'] < 100000]
-
-# Plotting
-fig, ax1 = plt.subplots(figsize=(11, 5))
-
-# Primary x-axis (time deltas)
-df['deltas'].plot(kind='hist', bins=100, ax=ax1)
-ax1.set_xlabel('Time-Stamp Deltas (us)')
-ax1.set_ylabel(f'Frequency (of {stamp_count})')
-
-# Secondary x-axis (bandwidth)
-ax2 = ax1.twiny()
-ax2.set_xlabel('Estimated Bandwidth (Mbits/s)')
-
-# Set the limits of the secondary axis based on the primary axis
-# new_tick_locations = np.linspace(df['deltas'].min(), df['deltas'].max(), num=len(ax1.get_xticks()))
-
-# Convert tick locations to bandwidth
-# bandwidths = [(pck_len * 8) * (1e6 / x) for x in new_tick_locations]
-
-x_ticks = ax1.get_xticks()
-bandwidth_ticks = [((pck_len * 8) * (1e6 / x)) / 1e6 for x in x_ticks]
-
-ax2.set_xlim(max(bandwidth_ticks), min(bandwidth_ticks))
-
-plt.title(f'Single-Source COBS Data Sink Deltas, pck_len={pck_len}')
-
-plt.tight_layout()
-
-plt.show()
+plot_stamps(stamps, stamp_count, pck_len, 0)
\ No newline at end of file

usb_serial/code/serial_source_teensy/.gitignore

+.pio
+.vscode/.browse.c_cpp.db*
+.vscode/c_cpp_properties.json
+.vscode/launch.json
+.vscode/ipch

usb_serial/code/serial_source_teensy/include/README

+
+This directory is intended for project header files.
+
+A header file is a file containing C declarations and macro definitions
+to be shared between several project source files. You request the use of a
+header file in your project source file (C, C++, etc) located in `src` folder
+by including it, with the C preprocessing directive `#include'.
+
+```src/main.c
+
+#include "header.h"
+
+int main (void)
+{
+ ...
+}
+```
+
+Including a header file produces the same results as copying the header file
+into each source file that needs it. Such copying would be time-consuming
+and error-prone. With a header file, the related declarations appear
+in only one place. If they need to be changed, they can be changed in one
+place, and programs that include the header file will automatically use the
+new version when next recompiled. The header file eliminates the labor of
+finding and changing all the copies as well as the risk that a failure to
+find one copy will result in inconsistencies within a program.
+
+In C, the usual convention is to give header files names that end with `.h'.
+It is most portable to use only letters, digits, dashes, and underscores in
+header file names, and at most one dot.
+
+Read more about using header files in official GCC documentation:
+
+* Include Syntax
+* Include Operation
+* Once-Only Headers
+* Computed Includes
+
+https://gcc.gnu.org/onlinedocs/cpp/Header-Files.html

usb_serial/code/serial_source_teensy/lib/README

+
+This directory is intended for project specific (private) libraries.
+PlatformIO will compile them to static libraries and link into executable file.
+
+The source code of each library should be placed in a an own separate directory
+("lib/your_library_name/[here are source files]").
+
+For example, see a structure of the following two libraries `Foo` and `Bar`:
+
+|--lib
+|  |
+|  |--Bar
+|  |  |--docs
+|  |  |--examples
+|  |  |--src
+|  |     |- Bar.c
+|  |     |- Bar.h
+|  |  |- library.json (optional, custom build options, etc) https://docs.platformio.org/page/librarymanager/config.html
+|  |
+|  |--Foo
+|  |  |- Foo.c
+|  |  |- Foo.h
+|  |
+|  |- README --> THIS FILE
+|
+|- platformio.ini
+|--src
+   |- main.c
+
+and a contents of `src/main.c`:
+```
+#include <Foo.h>
+#include <Bar.h>
+
+int main (void)
+{
+  ...
+}
+
+```
+
+PlatformIO Library Dependency Finder will find automatically dependent
+libraries scanning project source files.
+
+More information about PlatformIO Library Dependency Finder
+- https://docs.platformio.org/page/librarymanager/ldf.html

usb_serial/code/serial_source_teensy/platformio.ini

+; PlatformIO Project Configuration File
+;
+;   Build options: build flags, source filter
+;   Upload options: custom upload port, speed and extra flags
+;   Library options: dependencies, extra library storages
+;   Advanced options: extra scripting
+;
+; Please visit documentation for the other options and examples
+; https://docs.platformio.org/page/projectconf.html
+
+[env:teensy40]
+platform = teensy
+board = teensy40
+framework = arduino

usb_serial/code/serial_source_teensy/src/COBSUSBSerial.cpp

+// link ! 
+// TODO: pls use nanocobs, for encode- and decode-in-place, to save big (!) memory 
+// on new link layer... to fit into D11s... 
+
+#include "COBSUSBSerial.h"
+#include "cobs.h"
+
+
+#if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040)
+COBSUSBSerial::COBSUSBSerial(SerialUSB* _usbcdc){
+  usbcdc = _usbcdc;
+}
+#elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40)
+COBSUSBSerial::COBSUSBSerial(usb_serial_class* _usbcdc){
+  usbcdc = _usbcdc;
+}
+#else 
+COBSUSBSerial::COBSUSBSerial(Serial_* _usbcdc){
+  usbcdc = _usbcdc;
+}
+#endif 
+
+void COBSUSBSerial::begin(void){
+  usbcdc->begin(9600);
+}
+
+void COBSUSBSerial::loop(void){
+  // check RX side:
+  // while data & not-full, 
+  while(usbcdc->available() && rxBufferLen == 0){
+    rxBuffer[rxBufferWp ++] = usbcdc->read();
+    if(rxBuffer[rxBufferWp - 1] == 0){
+      // decoding in place should always work: COBS doesn't revisit bytes 
+      // encoding in place would be a different trick, and would require the use of 
+      // nanocobs from this lad: https://github.com/charlesnicholson/nanocobs 
+      size_t len = cobsDecode(rxBuffer, 255, rxBuffer);
+      // now we are with-packet, set length and reset write pointer 
+      // len includes the trailing 0, rm that... 
+      rxBufferLen = len - 1; 
+      rxBufferWp = 0;
+    }
+  }
+
+  // check tx side, 
+  while(txBufferLen && usbcdc->availableForWrite()){
+    // ship a byte, 
+    usbcdc->write(txBuffer[txBufferRp ++]);
+    // if done, mark empty
+    if(txBufferRp >= txBufferLen){
+      txBufferLen = 0;
+      txBufferRp = 0;
+    }
+  }
+}
+
+size_t COBSUSBSerial::getPacket(uint8_t* dest){
+  if(rxBufferLen > 0){
+    memcpy(dest, rxBuffer, rxBufferLen);
+    size_t len = rxBufferLen;
+    rxBufferLen = 0;
+    return len;
+  } else {
+    return 0;
+  }
+}
+
+boolean COBSUSBSerial::clearToRead(void){
+  return (rxBufferLen > 0);
+}
+
+void COBSUSBSerial::send(uint8_t* packet, size_t len){  
+  // we have a max: we need to stuff into 255, 
+  // and we have a trailing zero and the first key 
+  if(len > 253) len = 253;
+  // ship that, 
+  size_t encodedLen = cobsEncode(packet, len, txBuffer);
+  // stuff 0 byte, 
+  txBuffer[encodedLen] = 0;
+  txBufferLen = encodedLen + 1;
+  txBufferRp = 0;
+}
+
+boolean COBSUSBSerial::clearToSend(void){
+  // we're CTS if we have nothing in the outbuffer, 
+  return (txBufferLen == 0);
+}
+
+// we should do some... work with this, i.e. 
+// keepalives, to detect if other-end is open or not... 
+boolean COBSUSBSerial::isOpen(void){
+  if(usbcdc){
+    return true;
+  } else {
+    return false; 
+  }
+}
\ No newline at end of file

usb_serial/code/serial_source_teensy/src/COBSUSBSerial.h

+// example cobs-encoded usb-serial link 
+
+#include <Arduino.h>
+
+class COBSUSBSerial {
+  public: 
+    #if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040)
+    COBSUSBSerial(SerialUSB* _usbcdc);
+    #elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40)
+    COBSUSBSerial(usb_serial_class* _usbcdc);
+    #else
+    COBSUSBSerial(Serial_* _usbcdc);
+    #endif 
+    void begin(void);
+    void loop(void);
+    // check & read,
+    boolean clearToRead(void);
+    size_t getPacket(uint8_t* dest);
+    // clear ahead?
+    boolean clearToSend(void);
+    // open at all?
+    boolean isOpen(void);
+    // transmit a packet of this length 
+    void send(uint8_t* packet, size_t len);
+  private: 
+    #if defined(ARDUINO_ARCH_RP2040) || defined(ARDUINO_ARCH_RP2040)
+    SerialUSB* usbcdc = nullptr;
+    #elif defined(ARDUINO_TEENSY41) || defined(ARDUINO_TEENSY40)
+    usb_serial_class* usbcdc = nullptr;
+    #else 
+    Serial_* usbcdc = nullptr;
+    #endif 
+    // buffer, write pointer, length, 
+    uint8_t rxBuffer[255];
+    uint8_t rxBufferWp = 0;
+    uint8_t rxBufferLen = 0;
+    // ibid, 
+    uint8_t txBuffer[255];
+    uint8_t txBufferRp = 0;
+    uint8_t txBufferLen = 0;
+};
\ No newline at end of file

usb_serial/code/serial_source_teensy/src/cobs.cpp

+/*
+utils/cobs.cpp
+
+Jake Read at the Center for Bits and Atoms
+(c) Massachusetts Institute of Technology 2019
+
+This work may be reproduced, modified, distributed, performed, and
+displayed for any purpose, but must acknowledge the osap project.
+Copyright is retained and must be preserved. The work is provided as is;
+no warranty is provided, and users accept all liability.
+*/
+
+#include "cobs.h"
+// str8 crib from
+// https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing
+
+/** COBS encode data to buffer
+	@param data Pointer to input data to encode
+	@param length Number of bytes to encode
+	@param buffer Pointer to encoded output buffer
+	@return Encoded buffer length in bytes
+	@note doesn't write stop delimiter
+*/
+
+size_t cobsEncode(const void *data, size_t length, uint8_t *buffer){
+
+	uint8_t *encode = buffer; // Encoded byte pointer
+	uint8_t *codep = encode++; // Output code pointer
+	uint8_t code = 1; // Code value
+
+	for (const uint8_t *byte = (const uint8_t *)data; length--; ++byte){
+		if (*byte) // Byte not zero, write it
+			*encode++ = *byte, ++code;
+
+		if (!*byte || code == 0xff){ // Input is zero or block completed, restart
+			*codep = code, code = 1, codep = encode;
+			if (!*byte || length)
+				++encode;
+		}
+	}
+	*codep = code;  // Write final code value
+	return encode - buffer;
+}
+
+/** COBS decode data from buffer
+	@param buffer Pointer to encoded input bytes
+	@param length Number of bytes to decode
+	@param data Pointer to decoded output data
+	@return Number of bytes successfully decoded
+	@note Stops decoding if delimiter byte is found
+*/
+
+size_t cobsDecode(const uint8_t *buffer, size_t length, void *data){
+
+	const uint8_t *byte = buffer; // Encoded input byte pointer
+	uint8_t *decode = (uint8_t *)data; // Decoded output byte pointer
+
+	for (uint8_t code = 0xff, block = 0; byte < buffer + length; --block){
+		if (block) // Decode block byte
+			*decode++ = *byte++;
+		else
+		{
+			if (code != 0xff) // Encoded zero, write it
+				*decode++ = 0;
+			block = code = *byte++; // Next block length
+			if (code == 0x00) // Delimiter code found
+				break;
+		}
+	}
+
+	return decode - (uint8_t *)data;
+}

usb_serial/code/serial_source_teensy/src/cobs.h

+/*
+utils/cobs.h
+
+consistent overhead byte stuffing implementation
+
+Jake Read at the Center for Bits and Atoms
+(c) Massachusetts Institute of Technology 2019
+
+This work may be reproduced, modified, distributed, performed, and
+displayed for any purpose, but must acknowledge the osap project.
+Copyright is retained and must be preserved. The work is provided as is;
+no warranty is provided, and users accept all liability.
+*/
+
+#ifndef UTIL_COBS_H_
+#define UTIL_COBS_H_
+
+#include <Arduino.h>
+
+size_t cobsEncode(const void *data, size_t length, uint8_t *buffer);
+
+size_t cobsDecode(const uint8_t *buffer, size_t length, void *data);
+
+#endif

usb_serial/code/serial_source_teensy/src/main.cpp

+#include <Arduino.h>
+#include "COBSUSBSerial.h"
+
+#define PIN_LED 13 
+
+COBSUSBSerial cobs(&Serial);
+
+void setup() {
+  cobs.begin();
+  pinMode(PIN_LED, OUTPUT);
+  digitalWrite(PIN_LED, HIGH);
+}
+
+union chunk_uint32 {
+  uint8_t bytes[4];
+  uint32_t u;
+};
+
+chunk_uint32 chunk;
+
+uint32_t lastBlink = 0;
+
+void loop() {
+  // run comms
+  cobs.loop();
+  // tx a stamp AFAP 
+  if(cobs.clearToSend()){
+    chunk.u = micros();
+    cobs.send(chunk.bytes, 128);
+  }
+  // blink to see hangups 
+  if(lastBlink + 100 < millis()){
+    lastBlink = millis();
+    digitalWrite(PIN_LED, !digitalRead(PIN_LED));
+  }
+}

usb_serial/code/serial_source_teensy/test/README

+
+This directory is intended for PlatformIO Test Runner and project tests.
+
+Unit Testing is a software testing method by which individual units of
+source code, sets of one or more MCU program modules together with associated
+control data, usage procedures, and operating procedures, are tested to
+determine whether they are fit for use. Unit testing finds problems early
+in the development cycle.
+
+More information about PlatformIO Unit Testing:
+- https://docs.platformio.org/en/latest/advanced/unit-testing/index.html