Start a new topic

ChatGPT v Orba 1

ChatGPT v Orba 1

Part 1

Around page 22 of the "Orba hacking knowledge base", a year or so ago, me and @Subskybox were dissecting the eventData string the Orba 1 uses to represent sequences. @Subsky did some clever mathematical analysis while I did the donkey work of setting up experiments and recording the results.


Some of the experiments were based on a song called "DPC" which played the first seven notes of a minor scale. I've attached the song file, console output, and a spreadsheet @Subsky put together after analysing the data.

The eventData string is a mix of note and performance data, but this "DPC" test simplifies things to only include note data. This is organised as a series of "note blocks":

Note Block 1:

PlayNote: 16

startTicksLSB: 7

startTicksMSB: 0

Note #: 62

Vel On: 120

Vel Off: 90

DurTicksLSB: -11

DurTicksMSB: 1

Note Block 2:

PlayNote: 16

startTicksLSB: 89

startTicksMSB: 7

Note #: 64

Vel On: 127

Vel Off: 92

DurTicksLSB: -17

DurTicksMSB: 1

Note Block 3:


PlayNote: 16

startTicksLSB: -105

startTicksMSB: 7

Note #: 65

Vel On: 113

Vel Off: 92

DurTicksLSB: -46

DurTicksMSB: 3

Note Block 4:


PlayNote: 16

startTicksLSB: -122

startTicksMSB: 7

Note #: 67

Vel On: 121

Vel Off: 80

DurTicksLSB: -31

DurTicksMSB: 3

Note Block 5:


PlayNote: 16

startTicksLSB: 108

startTicksMSB: 7

Note #: 69

Vel On: 118

Vel Off: 58

DurTicksLSB: -91

DurTicksMSB: 1

Note Block 6:


PlayNote: 16

startTicksLSB: -100

startTicksMSB: 7

Note #: 70

Vel On: 127

Vel Off: 91

DurTicksLSB: -20

DurTicksMSB: 1

Note Block 7:


PlayNote: 16

startTicksLSB: 113

startTicksMSB: 7

Note #: 72

Vel On: 87

Vel Off: 55

DurTicksLSB: 116

DurTicksMSB: 1

If you take this series of values and encode them as a Base64 string, you get the corresponding following eventData string from the .song file:

"EAcAPnha9QMQWQdAf1zvAxCXB0FxXNIFEIYHQ3lQ4QUQbAdFdjqlAxCcB0Z/W+wBEHEHSFc3dAE="

This appears in the .song XML as follows:

<LoopData writeIndex="56" recordStartTime="0" recordStopTime="11882" lastEventTime="4809"

nBars="7" eventData="EAcAPnha9QMQWQdAf1zvAxCXB0FxXNIFEIYHQ3lQ4QUQbAdFdjqlAxCcB0Z/W+wBEHEHSFc3dAE="

eventDataCrc="1ff6d4c4"/>

The problem we found is that the timing data is relative...the timing of each note, ie when it plays, is affected by the one before. That makes real-time quantisation a bit of a nightmare. It might be posisble to implement "offline" quantisation, processing a .song file to quantise the data, or create new sequences based on MIDI data, but it's a hassle and we pretty much abandoned the investigation at that point.
 
A few months later, ChatGPT arrived on the scene...

 

 

 

 

 

 

song
(31.2 KB)
txt
(1.28 KB)
xlsx

1 person likes this idea

"Values 2 and 3 represent the absolute timing of the note in LSB/MSB form"

...correction...the RELATIVE timing.

The workflow is a bit of a mess, but I thought I'd document it for posterity FWIW while I still remember how it works.


MIDI files have a succession of note-on/note-off messages. I've been preparing them in Sonar, now Cakewalk by Bandlab, and was surprised when I found that my version of the DAW changed note-off to note-on with velocity 0 when exporting. Seems a bit odd to me, but apparently they're interchangeable in theory. The routines I've been getting ChatGPT to write expect that format.

There's a handy website called MIDI-Dump which is useful for analysing MIDI files.

https://github.com/g200kg/midi-dump

1) First I run "midi1.py furelise.mid". The program takes a MIDI file as a parameter and creates three files, "header", "notes" and "footer". Consider the first four lines of the output for "Fur Elise" in "notes":

76,71,4,4

76,0,240,244

75,38,0,244

75,0,240,484


Four values per line:

First value - MIDI note
Second value - Velocity
Third value -  Relative timing (ticks)
Fourth value - Running total or 'absolute' timing (ticks).

This is note-on, note-off (vel 0), note-on, note-off (vel 0).

2) Next routine is "durations.py". This reads notes.txt and creates durations.txt. First four lines:

76,71,4,4,240

76,0,240,244,0

75,38,0,244,240

75,0,240,484,0

It simply appends an extra value to each line - 0 for note-off (vel 0), duration for note-on (calculated as the difference between a note-on event and the corresponding note-off event which follows afterwards at some point; vel 0 for the same note value one or more lines later. Here we see that the duration for note 76 is 240; the time between the start of the note at 4 and the start of the ensuing note-off at 244.)

3) Next routine is "noteones.py". This reads durations.py and creates noteons.txt. First four lines:

76,71,4,4,240

75,38,240,244,240

76,57,240,484,240

75,70,240,724,240

This removes the note-off lines, while updating the relative timing of the remaining note-ons to take this into account. (MIDI files have relative timing data like the Orba, but with MIDI, these relative times reflect the note-ons with +ve velocity as well as the note-ons with 0 velocity which actually represent note-offs. Orba sequence data only uses the note-ons.)

So this four lines capture the first four notes of Fur Elise now; 76 75 76 75, together with velocity and  relative time. AT this point we can run...

4) dur2orb.py. This reads note-ons.txt and crunches it into Orba note-block form as "orba_notes.txt":

16,4,0,76,71,100,240,0

16,240,0,75,38,100,240,0

16,240,0,76,57,100,240,0

16,240,0,75,70,100,240,0

We now see the same four notes in the familiar (to me now) Orba note block format. 

16 announces the start of a note in Orba-speak

Values 2 and 3 represent the absolute timing of the note in LSB/MSB form

Value 4 is the note

Values 5/6 are vel-on, vel-off. I haven't bothered reading the vel-off values from the MIDI file; I just use a standard value of 100. Vel-off is oretty niche; who cares.

Values 7/8 are the duration in LSB/MSB format

(The LSB/MSB calculations are beyond me. I simply copied them out of a spreadsheet linked earlier that @Subskybox figured out and fed them into ChatGPT. It's all discussed in detail around page 20 or so of the Orba hacking thread.)

Finally, we can use rawtostring to convert this into an eventData string. This expects the input file to be called "note_blocks", not "orba_notes", which needs to be renamed.

We also need certain other values for the loopData XML for the furelise song file:

    <LoopData writeIndex="1536" recordStartTime="0" recordStopTime="41044" lastEventTime="41284"

              nBars="50" eventData="EAQA etc etc"

              eventDataCrc="e91384a5"/>

recordStopTime and lastEventTime were discussed previously; it's the absolute time plus duration of the final and penultimate events respectively. (Perhaps lastEventTime is really about when the last note finishes - which might not necessarily be when the last note in the sequence finishes, depending on duration. Not sure.) 

writeIndex, as mentioned earlier, is the number of values in the note_blocks file.

I haven't really thought about nBars much yet, I just set it high. It only matters for loops I guess.

And that's it, so far.

********************************************

So...why such a ragged, complicated procedure...? Well, I found it difficult to get ChatGPT to write some of this stuff. It certainly couldn't do it all in one go. So I broke it down into simpler steps. I was planning to then get ChatGPT to string them all together and streamline it, but I don't know if I'll bother. Certainly not yet.

song
(33.5 KB)
py
(807 Bytes)
py
(1.31 KB)
py
(1.11 KB)
py
(1.19 KB)
mid
(1.45 KB)
py
(1.38 KB)

The latest routines just enabled me to convert a polyphonic MIDI file of Für Elise into Orba XML and it played first time, no dummy data required - pleased with that. As well as writing the code, ChatGPT has been making helpful observations, such as identifying how the essential "writeIndex" value is calculated in the XML - it's simply the number of values in the note blocks file.

https://youtu.be/G-4zJExTIN0

...and ChatGPT just pointed out that the "recordStopTime" is similarly the time plus duration of the final note. 6678+179=6857.


OK, well, I don't know why the XML references those two sums, but it's handy to know how they're generated. That's a useful step towards creating a viable loopData entry from scratch, which is the next target...

5902 = 5714+188. That's actually the time plus duration of the penultimate note. Curious.

Here's a new test.

Song "Seq" has eight notes.

Loopdata is:

<LoopData writeIndex="64" recordStartTime="0" recordStopTime="6857" lastEventTime="5902"

nBars="4" eventData="EAoASmUpqQAQgANKTTCnABDjA0UzJLoAEJIDRTwnxQAQ4gNBbSzXABCyA0FwLK0AEL8DPlc2vAAQxAM+djqzAA=="

eventDataCrc="e91384a5"/>

stringtoraw.py:

16,10,0,74,101,41,169,0

16,128,3,74,77,48,167,0

16,227,3,69,51,36,186,0

16,146,3,69,60,39,197,0

16,226,3,65,109,44,215,0

16,178,3,65,112,44,173,0

16,191,3,62,87,54,188,0

16,196,3,62,118,58,179,0

decodeblocks.py:

MemIdx = 0 - MIDI Note at tick 10, channel 1, note 74, duration 169, von 101, voff 41

MemIdx = 8 - MIDI Note at tick 906, channel 1, note 74, duration 167, von 77, voff 48

MemIdx = 16 - MIDI Note at tick 1901, channel 1, note 69, duration 186, von 51, voff 36

MemIdx = 24 - MIDI Note at tick 2815, channel 1, note 69, duration 197, von 60, voff 39

MemIdx = 32 - MIDI Note at tick 3809, channel 1, note 65, duration 215, von 109, voff 44

MemIdx = 40 - MIDI Note at tick 4755, channel 1, note 65, duration 173, von 112, voff 44

MemIdx = 48 - MIDI Note at tick 5714, channel 1, note 62, duration 188, von 87, voff 54

MemIdx = 56 - MIDI Note at tick 6678, channel 1, note 62, duration 179, von 118, voff 58

This attempts to generate simulated "console data" from the Orba's built-in debug utility.

The real "console data" is:

         Using 64 of 4096 bytes of looper memory available (1 %)


         Looper Configuration:

                loopBarEndTolerance: 120

                beatLengthTicks: 480

                notesPerBar: 4

                quantMode: 0

                quantStartSnapTicks: 120

                quantBarEndSnapTicks: 240

                allowRecordingSilenceStart: 1

                allowRecordingSilenceEnd: 0

                thinnerMode: 1

                cc_error_limit: 1000

                pbend_error_limit: 25000

                ccA: 1

                ccB: 74

                ccC: 75

                thickenerMode: 1

                thickenerEmitPrd: 20

                thickenerMaxDt: 5000

                noteStartWindow: 240


MemIdx = 0 - MIDI Note at tick 10, channel 1, note 74, duration 169, von 101, voff 41

MemIdx = 8 - MIDI Note at tick 906, channel 1, note 74, duration 167, von 77, voff 48

MemIdx = 16 - MIDI Note at tick 1901, channel 1, note 69, duration 186, von 51, voff 36

MemIdx = 24 - MIDI Note at tick 2815, channel 1, note 69, duration 197, von 60, voff 39

MemIdx = 32 - MIDI Note at tick 3809, channel 1, note 65, duration 215, von 109, voff 44

MemIdx = 40 - MIDI Note at tick 4755, channel 1, note 65, duration 173, von 112, voff 44

MemIdx = 48 - MIDI Note at tick 5714, channel 1, note 62, duration 188, von 87, voff 54

MemIdx = 56 - MIDI Note at tick 6678, channel 1, note 62, duration 179, von 118, voff 58

 
...so that matches OK. 

Considering the rest of the XML:

LoopData writeIndex="64"
recordStartTime="0"
recordStopTime="6857"
lastEventTime="5902"

nBars="4"

Where does that come from, I wonder. Eg that figure of 5902 for lastEventTime. The last note is at 6678.  

py
song
(31.5 KB)

Replacing the MIDI notes in an existing sequence in this way is quite straightforward, and you can use the same approach to replace other data, timing, duration, velocity, etc. I'm currently trying to figure out how to generate an eventData string like this from scratch along with other relevant XML data, rather than by manipulating a 'dummy' string.

Corrected video link.

https://youtu.be/OGS634SKncE


More attachments...


py
(1.13 KB)
py
(807 Bytes)
py
(788 Bytes)

Here's a simple example to start introducing the scripts I've got so far. Start by finding a MIDI file - eg "Sweet Child O' Mine". So far I'm working on simple one-line examples with equally spaced notes.


 - Check the MIDI file in a DAW. (I'm using Cakewalk by BandLab on Windows. MP3 is scom.mp3, MIDI file is scom.mid.)


image

 
- Run "py miditovals-dedup.py scom.mid midi_notes.txt". All the Python routines were generated by ChatGPT. This one extracts the MIDI note values from the file and generates a simple comma-separated list. I've attached "miditovals.py" as well as "miditovals-dedpu.py". The reason for using the "dedup" version is that the regular version sometimes presents every note twice with certain MIDI files. This is probably because the original file contains both note-on and note-off and it reads them both or something.

 - Extract the value of the eventData string (everything within quotes) from the "Sequence" song file. This simply contains a long, rapid sequence of random approximately evenly spaced notes with no performance data. Copy the string into "note_string.txt".

 - Run "py stringtoraw.py". This converts the Base64 string into a file "note_blocks.txt" in this format:

16,103,0,64,80,48,45,0

16,121,0,64,106,63,45,0

16,114,0,64,108,61,44,0

...etc. These are the first three notes of the original Orba data, based on the structure:

PlayNote: (16)

startTicksLSB: 

startTicksMSB: 

Note #: 

Vel On: 

Vel Off: 

DurTicksLSB: 

DurTicksMSB:

 - Run "py tweaknotes.py". This substitutes the note values in note_blocks with the ones in midi_notes. In other words, the random notes are replaced by the ones from the MIDI file. Note that this approach will only work with a simple evenly-spaced single melody line. It generates "adjusted_note_blocks". Eg the Orba sequence now starts:

16,103,0,61,80,48,45,0

16,121,0,73,106,63,45,0

16,114,0,68,108,61,44,0

...where 61, 73, 68 are the first three notes of the sequence.

 - Replace the contents of "note_blocks" with this new adjusted data, and run "py rawtostring.py" to convert it back into a Base64 string. Make a copy of the .song file with suitable name/tempo and replace the value of eventData with this new string.

 - Upload to Orba:

https://youtu.be/0doaResDKJc

***********************************

(I've been working on other stuff but I'll present that separately.)

txt
song
(36.8 KB)
txt
(12.1 KB)
txt
(5.26 KB)
txt
py
(896 Bytes)
py
(745 Bytes)
mid
(10.4 KB)
mp3
(1.44 MB)

Now we just need a MIDI to Orba Base64 converter.. I'll dig up my old notes because I had mapped out a number of the command types (e.g. note = 16) but there were quite a few more data structures for pitch bend, cc control, etc...


1 person likes this

This one worked a bit better...Bach prelude. 


https://youtu.be/ZSkyBMTFmmw


1 person likes this

ChatGPT v Orba 1

Part 10

And here it is - Flight of the Bumblebee on an Orba 1. Almost. ;-)

https://youtu.be/wbUQEP_8NZE 


1 person likes this

ChatGPT v Orba 1

Part 9

It's not Flight of the Bumble Bee yet, but it's progress.

This is an MP3 recording of a 32-bar Orba sequence with a single-line melody provided by ChatGPT. I don't know what it is. I asked for "Toccata and Fugue" so it's probably Bach.

The method was:

- Record a 32-bar sequence with repeated rapid random notes and save it.
- Extract the eventData string
- Get ChatGPT to generate sequence of MIDI note numbers

I then used a set of Python utilites I've persuaded ChatGPT to write:

- Decode the string into note block data and export to a file
- Replace the notes with new ones which are read from a file
- Replace the durations with a set read from a file (all 100s here)
- Replace the timing information with a set of relative times read from a file (all 100s) 
- Encode the blocks back into a string

Finally I made a copy of the original song and replaced the eventData string with the new copy.

The next things I want to work out are being able to generate note sequence XML from scratch rather than having to modify recorded sequences, and working with expression data as well as note data.

Here's sample output from the Python routines.

Start of eventData string:

"EBAASnNEZAAQZABNYT5kABBkAFFzQWQAEGQAT31EZAAQZABNekNkABBkA"

Start of raw note block data:

16,16,0,74,115,68,100,0

16,100,0,77,97,62,100,0

16,100,0,81,115,65,100,0

16,100,0,79,125,68,100,0

16,100,0,77,122,67,100,0

16,100,0,74,92,51,100,0

16,100,0,76,92,56,100,0

16,100,0,74,102,51,100,0

16,100,0,77,102,52,100,0

16,100,0,79,92,51,100,0

16,100,0,80,76,48,100,0

Start of formatted note block data:

Note Block:

PlayNote: 16

startTicksLSB: 16

startTicksMSB: 0

Note #: 74

Vel On: 115

Vel Off: 68

DurTicksLSB: 100

DurTicksMSB: 0

Note Block:

PlayNote: 16

startTicksLSB: 100

startTicksMSB: 0

Note #: 77

Vel On: 97

Vel Off: 62

DurTicksLSB: 100

DurTicksMSB: 0

Note Block:

PlayNote: 16

startTicksLSB: 100

startTicksMSB: 0

Note #: 81

Vel On: 115

Vel Off: 65

DurTicksLSB: 100

DurTicksMSB: 0

MemIdx event strings (simulated "console output"):

MemIdx = 0 - MIDI Note at tick 16, channel 1, note 74, duration 100, von 115, voff 68

MemIdx = 8 - MIDI Note at tick 116, channel 1, note 77, duration 100, von 97, voff 62

MemIdx = 16 - MIDI Note at tick 216, channel 1, note 81, duration 100, von 115, voff 65

MemIdx = 24 - MIDI Note at tick 316, channel 1, note 79, duration 100, von 125, voff 68

MemIdx = 32 - MIDI Note at tick 416, channel 1, note 77, duration 100, von 122, voff 67

MemIdx = 40 - MIDI Note at tick 516, channel 1, note 74, duration 100, von 92, voff 51

MemIdx = 48 - MIDI Note at tick 616, channel 1, note 76, duration 100, von 92, voff 56

MemIdx = 56 - MIDI Note at tick 716, channel 1, note 74, duration 100, von 102, voff 51

MemIdx = 64 - MIDI Note at tick 816, channel 1, note 77, duration 100, von 102, voff 52

MemIdx = 72 - MIDI Note at tick 916, channel 1, note 79, duration 100, von 92, voff 51

mp3
(744 KB)

ChatGPT v Orba 1

Part 8

ChatGPT and its ilk tend to get a bit of a bad press.Most of the news articles now are about how soon it's expected to cause human extinction. You're also on thin ice when you start discussing programming. I'm aware that there are several talented programmers on this forum who I highly respect and wouldn't touch it with a bargepole. When people started using ChatGPT to try and generate code for Monome products and posted it on Lines (Monome community forum), they didn't get a warm welcome...

...nevertheless, I'm a fan. It's the arrival of ChatGPT that prompted me to re-engage with the Orba, and with programming in general, and I've found this quite rewarding.

To some extent you can engage with ChatGPT directly on this stuff, but it's very wayward and unpredictable, as you'll know if you've spent any time with it. It has flashes of brilliance but never does anything the same way twice. It's forgetful and makes basic errors. That's just its nature.

I've spent a number of hours now engaging with ChatGPT in discussions about note sequence data, and found the best approach for me was to use it to write Python routines. I like the idea of using ChatGPT as a human-language-level programming tool ("...create the Orba XML file to play Beethoven's Fifth on an ocarina...") but it's too frustrating to try and work that way, partly because it's so forgetful. It will give you the right answer one minute and the wrong answer the next. So I started working with it via Python instead. Because at least the action of a Python program, once it's been established, is predictable. It takes quite a lot of effort to get it to produce working code, and I expect it's not good code, but for a non-programmer it can nevertheless be a useful crutch. With patience.

I'll present the code at some point, after I've refined it. I've been developing routines to convert between Base64 eventData strings, raw "note block" data, formatted human-readable "note-block data"; routines to modify note data with new timing information, and interpret note data to produce "console output" with lists of notes and event times.

I was pleased with today's progress, and finally had a set of Python utilities that allowed me to create the XML to play a two-octave ascending and descending chromatic scale on Orba 1. Without ChatGPT, I would have found that so difficult that I wouldn't even have tried.

The challenge I've set myself is to sequence the opening of "Flight of the Bumble Bee". Unfortunately ChatGPT doesn't  seem familiar with the note data for it so I'll have to round it up from somewhere else. If I can get that far I'll post up a YT link and some code. 

Login or Signup to post a comment