- Cuttlefish
- The Latest
- w/r/t force layout
- Considering Force Layout
- Traversing Heirarchy
- Scratch
- Bugs
- Program 'Recipes'
- Loading, Saving Patches
- Important Lessons Learned
- Views are Hunks, Too
- State in the DOM; use JQuery & Unique Human-Readable IDs
- Give Up on Storing Position
- CSS Transforms are a PITA
- JS Promises are Great
- End Notes
- UI / Usability
- Videos to Take / Demos
- Known Bugs
- Bug Foreshadowing
- View Potshots / UI Code Golf
Cuttlefish
Cuttlefish is a Distributed Dataflow Machine Control Context for the browser.
Cuttlefish is an 'Integrated Development and Operation Environment' or something - an IDOE (?) / IDDE: an 'Integrated Dataflow Development Environment'.
It's main job is to provide views (see hunks/hidden/view.js) into other controllers' brains, and help you compose those brains, hook them up to eachother, and make sense of big sprawling programs as one big pile of delicious messaging spaghetti.
Native Cuttlefish 'Hunks' are given access to a DOM/div element, making them nice UI canvases.
The Latest
I'm excited about this, so here's an image:
... many lines of js later, I can 'explode' hunk definitions, which means I can invert link relationships, so that I can visually explain nested network relationships in the graph. I've also simplified a lot of view objects, and have come up with this satisfying idea about recipes and atomic operations that I enjoy - that helps me a lot in patch loading, saving, etc - especially for big messy patches (I hope)
w/r/t force layout
I've re-written much of the display code. the 'view' hunk, that does rendering work, is about 2500 lines all together at this point, which is a pretty yikes-level of code for me.
That said, it's basically the whole project's workhorse - which is interesting in itself. This is one thing I'm definitely excited about for the project: the contexts / managers are very simple - the whole thing has this pretty straightforward 'hind brain' that is just pushing memory around (inside of contexts) and pushing bytes around (over the network).
This might sound like a boast, but I guess this is kind of similar to writing a programming language ... at the base, the instruction set is really small. The syntax, rules, compiling, etc of the language is big, cumbersome, and messy.
The most recent add is to do with making visual sense of nested programs. In order to do this, I've broken up what were previously single <div>
elements into collections of them... i.e. inputs, outputs, and state each get a box to live within. This causes some more complexity to the view program because I have to keep track of these handfuls of html elements, and then there's this: I use the same mechanism to 'explode' hunk-definitions, and to 'wrap' (i.e. links) around others (i.e. views). This is really handy, because it lets me accurately draw link->link representations 'across' the link divide, and lets me 'unwrap' a pattern that was very visually cluttered: this common 'hourglass' structure (i.e. feedback).
So, I wrote this all in a rush, in what felt like a javascript bender, and now I'm trying to catalogue accurately what-all happened, to track and hopefully resolve. What I'm working towards is bringing force layout back into the picture, because (as is easy to tell when opening a big program) these things are hot hot messes, and I'm not up to storing positions exactly yet, because that sounds counter to the point.
Here's the things I'd like to be able to do:
- blow-up definitions, having inputs, outputs, and cores be in potentially different places.
- have 'exploded' elements, probably most often used to 'unwrap,' say, a data pipe around a link, (to unconfuse the hourglass)
- using exploded elements, pin outputs to the left, and inputs to the right. this in particular to show a link 'across' a view
- wrap a link around a view
- resize views, and other 'native' elements.
Other tricks
- the message board should perhaps belong only to the top-level view, and should be easily hidden
Considering Force Layout
There'll be a handful of tricks to make this work out well
-> a good update model -> this thing can also check our 'rulesets' per the heirarchy, for links and views and whatnot -> that's a kick (topology changes and checks), and a tick (simulation step) -> integrating this with manual movements ... -> the floop should probably only run at the top level ... but subroutines will be nice per view because -> we will have / want to adjust the size of sub-views (in a bubbling kind of way) from the bottom-up... i.e. we really want to avoid overlap, etc
Probably a good first step is a run through the .def tool, to invent a decent API for moving, exploding, gathering, etc. making 'edge' types. Three types...
- fixed
- exploded (just str8 up kapow, but maybe as a rule just blow up the input ?))
- wrapped
- unwrapped
For each object, I'll want to re-write the .deg api so that .deg.inputs.moveto() is a viable function, etc ... such that when I'm running through the sim, every free entity can be one of these ->
floater.moveto(x,y) // floater.fix(x,y)
and those can be walked as nodes ...
dragging objects ? just kill the sim I guess ? or fix it during the sim ... dragging objects should use the same floater.moveto() setup, or a .moveby()
then I also have to handle native-element resizing ... so .wrap objects need to react to that...
and then there's this question about how to do it heirarchichally - or all at once, in one floop ? maybe all at once is easier ...
A good way to set rules ... like, what's fixed etc. I.E. consider -> the ears of a link expand 'around' a program, pushing the view definition 'out' - how will these 'forces' be connected? Just .set the view to match, let them float? Pin the left to 0,0 and float the right? Then take the lower level constraints as fact? nice.
Traversing Heirarchy
I am having a feeling that I am going about this poorly. I should remember that I am here to build a tool, not get into semantics about UI and write nice APIs for myself. Alas, black holes abound.
I need to get to ponyo.
I want to open programs with heirarchy. I'm close to doing this but I have to suss out this relationship between the top-layer view and the lower-level(s) view(s).
I think my issue is that I'm just kind of trammeling through this without any strong ideas about how the larger structures will work. Lower level things feel OK so far.
On a lack of heirarchy sweeping... I am curious to keep the recepies to a minimum. Surely, I'll save systems with heirarchy. When I'm loading them, I can be intelligent about how to make the dives. Once I load a layer, any links (with programs defined beneath them) I can interrogate. I can write a routine to hunt (using link-traversal) for connected views. If those views are available, I'll connect those views to the link (definition?) and use those to traverse another layer. Then I can do program->link.view->defs... as a reference structure. This means that, yes, I should attach (to link defs) view (hunks). Which may get somewhat confusing. However!
This handles program traversal. What else I need is to wrap on (hah) my implementation of these force layout loops. Critically, I need events to 'bubbble' through them.
My missing link, and maybe the only thing I haven't really api'd well, is the resize-ability of elements. As a hack, I wrote this into my domtools tool ... which leaves things somewhat ambiguious. I need to call a resize: (1) on ui events, and (2) internally - i.e. a view(hunk) or any other native(hunk) should be able to call .requestResize(w,h). Any time the resulting element.resize(w,h) fn is called, a change should be triggered: this layer's floop should run from alpha(1) (making sure it has a bigger r, now). This floop, I think, will (in a few cases) also call it's domain's .requestResize(w,h) event - so, that's the link through.
In the same vein, a lower level view has a key difference from the higher level view, in that an element (floater) moving around inside, should not be able to move past the 'wall' of the view's plane. I can probably do this in a few ways, (1) by just writing in limits... but I will have to differentiate between lower-level views and higher-level views. I can also modify lower- and upper- level floops to incorporate (or not) a force at the edges. This would be for an investigation into the floop's internals (i.e. the D3 force library).
Scratch
So, to start, I'll
-
bounds limits proper
-
bounds expansion by request?
-
get top level scale for drag moves
-
want zoom-to-extents
-
open internal
-
bring floop to lower level, how ?
-
if each has a floop,
-
top level has a loop
-
at each loop, do views
-
internal operators need to check bounds, external should zoom to extents, or have button to do ...
-
want on view open for the color-dive to happen
-
context menu should be sensitive to location: can't load or merge patch from lower level
-
cleanup ... refresh doesn't clear floaters
-
links really don't want to operate from the center of a node ... fixed groups ?
-
once this, open merge program, etc, and push link internal to edges
-
will have to handle multi-view case ...
-
then go for expanding 2nd link ...
-
b4 ponyo do a writing session
-
ok
-
want some rules
-
floop resets on:
-
link add / rm
-
def add / rm
-
def explode / gather / etc
-
floop restarts on
-
drag (a drag move auto-fixes)
-
fix / unfix
-
escape from the floop
-
add button ... expand recipe
-
how to add natives,
-
how do resizes work ?
-
lower-level hunk/native dom elements have a 'this.domResizeRequest(x,y)' move ... and an init value like this.requestResizeableDom = true
-
then do .wrapOn() code
-
then do .unwrap() code
-
def.unwrap(), def.wrapon(floater) etc
-
to come; def.edgecase() ... has some relationship to a view, and its extents, and its resize ...
-
can you merge a patch yet ?
-
ok
-
find link ... adding link state -> .isConnected, .gatewayIndex
-
link.makeDoorway() ... after refresh ?
-
plane overflow ?
-
links expand need to find toplevel view object
-
ui elements are on their way
-
needs a rework somehow ... design a scheme, etc
-
I think this comes after handling the message panel and the .makeDoorway link business...
-
overlay, probably want to modify the message panel ...
-
... -> to space
-
writing it all up, and proceeding to a ponyo link
-
after a merge, do unwrapping ...
-
ok, ok,
-
then to nautilus, to look at unwrapping and matching on internal link
-
this will mean (sometimes) adding a new route through the link, that's a move
-
patches ... still save, do link newfangled state for matchup
-
top level merge system,
-
rewrite small policies after refresh as .invert() call to .def ? or .insideout()
-
native hunks add
-
views nest into links, exploded ...
-
native hunks add, button add, to .deg ... and then move moves them all about ... or just cases for buttons and natives ...
-
views get a 'green' header like inputs / outputs ... and we do .wrap(view.hunk.dom) on these
-
link expansion button, a button
-
side panel exists on another plane / can collapse /
-
link walls, etc
ok, looking out to more advanced UI stuff
- want to be able to 'cut' a def into inputs, outputs, and state items
- still one 'def' ... separate, floating def.de ...
- should be indicated by a collateral highlight
the case of the view dom element ... wants to have resize handles, demo this first
some thoughts:
-
'cuttlefish hunks' also only belong in toplevel view,
-
view wants / needs internal handles to itself, perhaps should be just one hunk ...
-
it's possible to do this without ever saving, or referencing a view, except for the fact that they're routed objects as well, links need to leave room for them
-
when opening,
-
link exists
-
view appears,
-
routes connect
-
then we can draw
-
considering more link complexity: when a link is NC on the other end, inputs to it should vanish into aether, same as an nc output in a program, no? this also means we don't have to keep track of whether a manager is hooked up anywhere or not
Bugs
- adding cuttlefish hunks / dom elemengs is borked, recreate this:
- add a link
- add a view
- remove the view
- add a new link ...
- something very up with deleting stuff,
'aye, but as the legends state: software doth make the hardware singeth'
Program 'Recipes'
Notes below about loading and saving patches walk into this note.
What I've just done is refactored all view program-editing related requests as promises, making it easy to chain async requests with eachother. I have 8 core message types, here:
VIEW REQUEST | Args | MGR REPLY | Args |
---|---|---|---|
Hello | - | Hello | - |
Query | - | Brief | <int name (str), int version (str), num hunks (uint16)> |
Query | <index (uint16)> | Hunk Definition | <...> |
Req List Avail | ... <opt, future, prefix (string)> | List Avail | <n items (string)s> |
Req Add Hunk | <type (string)> | Hunk Definition | <...> |
Req State Change | <hunk index (uint16), state item index (uint8), value (type)> | Hunk State Change | ~ same args ~ |
Req Rm Hunk | <index (uint16)> | Hunk Removed | <index (uint16)> |
Req Add Link | <output hunk index (uint16), output index (uint8), input hunk index (uint16), input index (uint8)> | Link Alive | ~ same args ~ |
Req Rm Link | <output hunk index (uint16), output index (uint8), input hunk index (uint16), input index (uint8)> | Link Rm'd | ~ same args ~ |
These are my atomic requests, and I can format program saving and loading as collections of them. When these are promises, that's easy to write - the promises are really handy state tracking syntax elements.
So, one recipe to start with is merging a program that is running with something that is being loaded. To start on this, I'll make my refresh() recipe itself into a promise, then I can start cooking with pre-prepared operations.
The next would be, from a link, expanding to a view. This would be a recipe that loads a new view, connects it to the link's 0th ports, and refreshes that view.
Using that as a tool / sub-recipe, I can imagine heirarchichal loading, done all async like. ok,ok,ok
Loading, Saving Patches
There's one layer missing to this whole thing - w/r/t saving and loading patches.
My earliest approach of this was to have each manager capable of loading and saving representations of programs. This makes sense, and is handy for startup, etc. However, it's quite cumbersome for remote managers, especially if we want to be sensitive to embedded programs.
There's another interesting approach, which is to remove all 'program' representations from the contexts themselves and bring it all together in the 'views' - to cuttlefish.
This has a few advantages, and some disadvantages. The largest of which is that it decreases the overall system complexity, mostly by bringing much of the state-related work into one location, where it can be reliably tracked. This also greatly reduces the list of messages which a manager should be tasked with operating over. When we bring 'patches' into cuttlefishes' domain, we can assemble programs (patches = programs) (collections of hunks and links) by sequentially requesting that hunks, and then links, be added to a remote manager.
This also means that we don't have to save anything to a remote node...
An added bonus is that we can save cumbersome UI data on top of patches: graphical layout, etc, of programs. And user state, like whether a number input should be a slider or not, or have bounds, etc. Or the size of a text input. These all seem rather handy.
This is also hugely helpful because we are going to have to (or at least want to) make an attempt to 'load' programs that operate across multiple managers. The only place we really know about the relationship between one managers' program and another is inside of the view... or, across the link (another option).
The disadvantage here is singular, but feels big. This is that we can't necessarily run anything without a view to start things up. That is, we bring global program init into a central location... so we can't power cycle systems. Imagine (as will be common) the desire to use cuttlefish as a kind of pickaxe: to assemble controllers for things like drones, walking robots, and (generally) things whose controllers dont serve an interface: controllers that listen only (or at least mostly) to the outside world, and respond to that. In these cases, we'll want the things to start up with some state: that is, with some program running.
There might be a way to put these two worlds together. That is, for remote managers to keep a 'stash' of (only) the program they are currently running, and to run those on bootup. We can write view-init-and-load tools that can be sensitive of local state, querying (1st) for a # of running hunks, and links, and querying through those as / before they load on top of them.
To walk in on this problem, I should try to consider how I might hope to represent these scattered programs...
- do links nest under hunk outputs, or are they separate? the manager does it one way, but this is incompatible with the sequential-loading ... unless sequential loading is just written a bit differently!
- how does heirarchy work, across links and across 'data ports' (naming those?) - which are flat and which are nested?
Important Lessons Learned
Today I'm combing through my very rough notes taken while I wrote this thing. I take for granted that it's not done yet, but in the effort of spiraling through writing... Here are some synopsis that I should not forget are things-worth-mentioning.
Views are Hunks, Too
An important aspect was that the 'View' (because we can have many of them) is also a 'Hunk' - this was at times confusing when writing the 'native' 'manager' for the browser, but is going to be a real watershed when I start opening offworld contexts up. This way nothing is mysterious - views aren't mystery windows into other worlds, they are software objects whose messages are obviously routed through the framework.
The UI participates in the computation: anything a user does needs also to be flow-controlled, passed as a message, etc.
This means (as I hope I will see) I can route many views through many contexts, although the view(s) are still obviously (as they are) hunks that operate in the cuttlefish context only.
This still annealing, but I like the direction. It's a tricky pickle, and really at the heart of this problem. Singular views into many programs running together, at once.
State in the DOM; use JQuery & Unique Human-Readable IDs
A big windfall for implementation was my adoption of JQuery and DOM-element-id-writing for selection, etc. This makes programming kind of 'conversational' ... I think most web developers know this, but now I do to. This lets me avoid keeping some global state objects, instead just inspecting the DOM every time something happens and checking it for consistency against messages I receive from the manager.
Give Up on Storing Position
TBD, but I'm going to try to adopt a policy of live-positioning everything. D3 has a great, straightforward force-layout scheme... I can use this to load graphs and draw them in 2D space (actual dimensionality: ?) without having to write those positions into the program representations, where they are meaningless unless rendered.
CSS Transforms are a PITA
I spent a good deal of time struggling with scaling and panning views. I'm not sure that I learned anything, and while I couldn't find it at the time, I think I have figured out a simple way to alleviate my pain on this front. I'll leave this note for myself here: div(div).
JS Promises are Great
For async module loading, etc. Do use. Herringbones nicely with event graph model and is a helpful model when considering how message interperetation should work (i.e. forces / reminds me to consider message reception as async).
End Notes
UI / Usability
At the moment, the UI really hobbles along. There are a few big bugs and problems to undo and I've plans to crush these out apres serialization is 'solved'.
- a session with D3 ... to make it genuinely work
- this goes along with bz tools
- debug why BZ tools fails with a nested view (just... integrate taht state with view... carefully pick through)
- this also goes along with 'view methods' ... poking through the graph and enforcing adjacencies between views, links, and data devices ... and the highlighting through
- try just the highlight through the link -> find by name, this leads to finding and positioning
Videos to Take / Demos
- loading with view relaxation
- loading a link, spawning a view
- show two cuttlefishes with views into the same nautilus, editing together... or just pushing program edits back and forth.
Known Bugs
- adding link from the DOM doesn't work when id's don't have one underscore in them... (is this still true?)
- an error while adding a link prevents a program (or hunk) from being sent to the view ... addHunk and addLink errors should be carefully handled
Bug Foreshadowing
Here's a list of things that I suspect will eventually roll themselves into bugs with enough testing.
- receiving messages from tree crawl on right-click, after menu dom has already been cleared
- many errors are not appearing when things fail to work ... I believe any undefined / null etc are lost in the wind somewhere. try writing to an object that doesn't exist, or returning it...
- still a drift on the y-axis when zooming // difference between background positioning and div positioning, perhaps div transform origin is lost ?
- program load browser does not consider the case for two identical supplied module ids - i.e. loading two programs (i.e. copy/paste section !)
View Potshots / UI Code Golf
- link hookup still doth not rock
- contexmenu click not localized (when zoomed out)
- would like state to be size-by-value ... for strings ... w/r/t names
- escape key to leave menu (/ keys on dom ? /on new right-click etc)
- on zoom, check if element is view or not,
- if not view (is block) then look for event parent position, not event