Real Guitar module POC

[sbeuh]

Hi,

I'm opening this thread cause I just finished a POC code that allows me to play pro guitar tracks of Rockband 3 on phase shift with my real guitar and a rocksmith real tone cable (no midi hardware needed).

Currently I'm using an audio to midi converter software and a piece of code that convert every midi notes to Sysex messages.
As we can't currently make a real difference between strings without midi hardware (ex: A string 5 fret with D string). I'm cheating by sending several sysex messages at once.

This is working but it's dirt.

I was wondering if real guitar in midi settings have a simple midi recognition method (based on midi notes sent by the midi input (like for drums) or only sysex messages. I tried with <method>X</method> (X from 1 to 3) But I only got results with method 2 that seems to be the Sysex one.

I saw that the last update of phase shift is really old but I see a nice activity on the forum. I was wondering if the project is still alive or if you miss some devs. If needed, I would love to help you with some features. You have the only game that allow almost an entire band to play together with real instruments.

I also worked on a karaoke software previously, seeing how advanced is your game I think that It should not be that hard to integrate some voice recognition code!

Thanks for reading.
musically,

[David]

Hi I'm not sure exactly what you're after but this code from the game might help you figure out what the games doing with the MIDI events.


#define RAW_TYPE_GAMECONTROLLER (0)
#define RAW_TYPE_KEYBOARD (1)
#define RAW_TYPE_MOUSE (2)

#define CTRL_BUTTON_COUNT (29)
#define CTRL_REPEAT_BUTTON_COUNT (20)

#define CTRL_DRUMS (0)
#define CTRL_GUITAR (1)
#define CTRL_GAMEPAD (2)
#define CTRL_KEYBOARD (3)
#define CTRL_KEYS (4)
#define CTRL_DRUMS_REAL (5)
#define CTRL_GUITAR_REAL (6)
#define CTRL_DANCE_PAD (7)

#define CTRL_TYPE_COUNT (8)

#define CTRL_BUTTON_UNKNOWN (0)
#define CTRL_BUTTON_GREEN (1)
#define CTRL_BUTTON_RED (2)
#define CTRL_BUTTON_YELLOW (3)
#define CTRL_BUTTON_BLUE (4)
#define CTRL_BUTTON_ORANGE (5)
#define CTRL_BUTTON_PURPLE (6)
#define CTRL_BUTTON_YELLOW_2 (7)
#define CTRL_BUTTON_BLUE_2 (8)
#define CTRL_BUTTON_GREEN_2 (9)
#define CTRL_BUTTON_BASS (10)
#define CTRL_BUTTON_BASS_2 (11)
#define CTRL_BUTTON_STRUMUP (12)
#define CTRL_BUTTON_STRUMDOWN (13)
#define CTRL_BUTTON_UP (14)
#define CTRL_BUTTON_DOWN (15)
#define CTRL_BUTTON_LEFT (16)
#define CTRL_BUTTON_RIGHT (17)
#define CTRL_BUTTON_START (18)
#define CTRL_BUTTON_BACK (19)
#define CTRL_BUTTON_SLIDER (20)
#define CTRL_BUTTON_WHAMMY (21)
#define CTRL_BUTTON_XGYRO (22)
#define CTRL_BUTTON_YGYRO (23)

#define CTRL_BUTTON_YELLOW_CLOSED (24)
#define CTRL_BUTTON_YELLOW_OPEN (25)
#define CTRL_BUTTON_YELLOW_PEDAL (26)
#define CTRL_BUTTON_YELLOW_SIZZLE (27)
#define CTRL_BUTTON_RIM (28)


#define CTRL_BUTTON_STRING_1 (35)
#define CTRL_BUTTON_STRING_2 (36)
#define CTRL_BUTTON_STRING_3 (37)
#define CTRL_BUTTON_STRING_4 (38)
#define CTRL_BUTTON_STRING_5 (39)
#define CTRL_BUTTON_STRING_6 (40)

#define CTRL_BUTTON_STRING_1_FRET (41)
#define CTRL_BUTTON_STRING_2_FRET (42)
#define CTRL_BUTTON_STRING_3_FRET (43)
#define CTRL_BUTTON_STRING_4_FRET (44)
#define CTRL_BUTTON_STRING_5_FRET (45)
#define CTRL_BUTTON_STRING_6_FRET (46)

#define CTRL_BUTTON_SYSDATA (50)

#define CTRL_BUTTON_KEY_BASE (100)
#define CTRL_BUTTON_KEYTAR_BASE (148)

#define CTRL_BUTTON_KEY_GREEN (0)
#define CTRL_BUTTON_KEY_RED (2)
#define CTRL_BUTTON_KEY_YELLOW (4)
#define CTRL_BUTTON_KEY_BLUE (5)
#define CTRL_BUTTON_KEY_ORANGE (7)

#define CTRL_BUTTON_KEY_GREEN_ALT (0)
#define CTRL_BUTTON_KEY_RED_ALT (1)
#define CTRL_BUTTON_KEY_YELLOW_ALT (2)
#define CTRL_BUTTON_KEY_BLUE_ALT (3)
#define CTRL_BUTTON_KEY_ORANGE_ALT (4)

#define CTRL_BUTTON_KEY_GREEN_ALT_2 (4)
#define CTRL_BUTTON_KEY_RED_ALT_2 (6)
#define CTRL_BUTTON_KEY_YELLOW_ALT_2 (8)
#define CTRL_BUTTON_KEY_BLUE_ALT_2 (10)
#define CTRL_BUTTON_KEY_ORANGE_ALT_2 (11)
//XINPUT Additions
#define XINPUT_DEVSUBTYPE_GUITAR_2 0x07
#define XINPUT_DEVSUBTYPE_GUITAR_REAL 0x19
#define XINPUT_DEVSUBTYPE_BASS 0x0B
#define XINPUT_DEVSUBTYPE_KEYS 0x0F
//Real Guitar Buttons
#define MIDI_GUITAR_BUTTON_BLUE 1
#define MIDI_GUITAR_BUTTON_GREEN 2
#define MIDI_GUITAR_BUTTON_RED 4
#define MIDI_GUITAR_BUTTON_YELLOW 8
#define MIDI_GUITAR_BUTTON_BACK 1
#define MIDI_GUITAR_BUTTON_START 2
#define MIDI_GUITAR_BUTTON_XBOX 16
#define MIDI_GUITAR_BUTTON_SYNC 64
#define MIDI_GUITAR_BUTTON_TILT 64
#define MIDI_GUITAR_BUTTON_UP 0
#define MIDI_GUITAR_BUTTON_RIGHT 2
#define MIDI_GUITAR_BUTTON_DOWN 4
#define MIDI_GUITAR_BUTTON_LEFT 6
#define MIDI_GUITAR_BUTTON_CENTER 8

//Controller Methods For Non XInput
#define CTRL_METHOD_MANUAL 0
#define CTRL_METHOD_REAL_DRUMS 1
//MIDI Methods
#define MIDI_METHOD_MANUAL 0
#define MIDI_METHOD_KEYBOARD 1
#define MIDI_METHOD_GUITAR_SYSEX 2
#define MIDI_METHOD_DRUMS 3
#define MIDI_METHOD_GUITAR_MIDI 4
#define MIDI_METHOD_LEGACY_GUITAR_MIDI 5


bool MidiCtrl::Method( MidiInEvent * InEvent, CtrlEvent *Data, int *DataCount, int BindID, int Device )
{
if( ( MidiConfigList[BindID].Method == MIDI_METHOD_DRUMS ) && !InEvent->sysex_count )
{
//CC#4 HiHat Ctrl
if( ( InEvent->type == 11 ) && ( InEvent->note == 4 ) )
AddEventEx( CTRL_BUTTON_BASS_2, Data, DataCount, InEvent->value, 127, 0, true );
}
//Midi Keyboard
if( ( MidiConfigList[BindID].Method == MIDI_METHOD_KEYBOARD ) && !InEvent->sysex_count )
{
if( InEvent->type == 14 )//Pitch Wheel
AddEventEx( CTRL_BUTTON_WHAMMY, Data, DataCount, InEvent->value, 128, 0, false );
else
AddEvent( CTRL_BUTTON_KEY_BASE + InEvent->note, InEvent, Data, DataCount, NULL );
}
//Midi Legacy Guitar Whammy
if( ( MidiConfigList[BindID].Method == MIDI_METHOD_LEGACY_GUITAR_MIDI ) && !InEvent->sysex_count )
{
if( InEvent->type == 14 )//Pitch Wheel
AddEventEx( CTRL_BUTTON_WHAMMY, Data, DataCount, InEvent->value, 128, 0, false );
}
//Midi Guitar
if( ( MidiConfigList[BindID].Method == MIDI_METHOD_GUITAR_MIDI ) && !InEvent->sysex_count )
{
if( InEvent->type == 9 )//Note On
{
int Fret = 0;
switch( InEvent->channel )
{
case 0: Fret = InEvent->note - 64; break;//E High
case 1: Fret = InEvent->note - 59; break;//B
case 2: Fret = InEvent->note - 55; break;//G
case 3: Fret = InEvent->note - 50; break;//D
case 4: Fret = InEvent->note - 45; break;//A
case 5: Fret = InEvent->note - 40; break;//Low E
}
//Change Fret
AddEventEx( CTRL_BUTTON_STRING_6_FRET - InEvent->channel, Data, DataCount, Fret, 27, 0, false );
//Strum
AddEventEx( CTRL_BUTTON_STRING_6 - InEvent->channel, Data, DataCount, InEvent->value, 127, 0, false );
}
}
//Sysex Guitar
if( ( MidiConfigList[BindID].Method == MIDI_METHOD_GUITAR_SYSEX ) && ( InEvent->sysex_count > 7 ) )
{
//Sysex Midi Signal Check
if( ( InEvent->sysex_data[0] != 0xF0 ) ||
( InEvent->sysex_data[1] != 0x08 ) ||
( InEvent->sysex_data[2] != 0x40 ) ||
( ( InEvent->sysex_data[3] != 0x08 ) && ( InEvent->sysex_data[3] != 0x0A ) ) )
return false;

char SysexType = InEvent->sysex_data[4];

if( SysexType == 1 )//Frets
{
int string = int( InEvent->sysex_data[5] );
int fret = 0;
switch( string )
{
case 1: fret = ( int( InEvent->sysex_data[6] ) - 64 ); break;
case 2: fret = ( int( InEvent->sysex_data[6] ) - 59 ); break;
case 3: fret = ( int( InEvent->sysex_data[6] ) - 55 ); break;
case 4: fret = ( int( InEvent->sysex_data[6] ) - 50 ); break;
case 5: fret = ( int( InEvent->sysex_data[6] ) - 45 ); break;
case 6: fret = ( int( InEvent->sysex_data[6] ) - 40 ); break;
}
AddEventEx( CTRL_BUTTON_STRING_6_FRET - ( string - 1 ), Data, DataCount, fret, 27, 0, false );
}
if( SysexType == 5 )//Strings
{
int string = int( InEvent->sysex_data[5] );
int value = int( InEvent->sysex_data[6] );
AddEventEx( CTRL_BUTTON_STRING_6 - ( string - 1 ), Data, DataCount, value, 127, 0, false );
}
if( SysexType == 8 )//Buttons
{
char CurrentButtonState[4];
char Changes[4];

for( int i = 0; i < 4; i++ )
{
CurrentButtonState[i] = InEvent->sysex_data[5+i];//Get New Sates
Changes[i] = ( CurrentButtonState[i] ^ MidiInput[Device].LastButtonState[i] );//Get Changes
}
//Set 1
if( Changes[0] & MIDI_GUITAR_BUTTON_BLUE )
AddEventEx( CTRL_BUTTON_BLUE, Data, DataCount, CurrentButtonState[0] & MIDI_GUITAR_BUTTON_BLUE, 128, 0, false );
if( Changes[0] & MIDI_GUITAR_BUTTON_GREEN )
AddEventEx( CTRL_BUTTON_GREEN, Data, DataCount, CurrentButtonState[0] & MIDI_GUITAR_BUTTON_GREEN, 128, 0, false );
if( Changes[0] & MIDI_GUITAR_BUTTON_RED )
AddEventEx( CTRL_BUTTON_RED, Data, DataCount, CurrentButtonState[0] & MIDI_GUITAR_BUTTON_RED, 128, 0, false );
if( Changes[0] & MIDI_GUITAR_BUTTON_YELLOW )
AddEventEx( CTRL_BUTTON_YELLOW, Data, DataCount, CurrentButtonState[0] & MIDI_GUITAR_BUTTON_YELLOW, 128, 0, false );
//Set 2
if( Changes[1] & MIDI_GUITAR_BUTTON_BACK )
AddEventEx( CTRL_BUTTON_BACK, Data, DataCount, CurrentButtonState[1] & MIDI_GUITAR_BUTTON_BACK, 128, 0, false );
if( Changes[1] & MIDI_GUITAR_BUTTON_START )
AddEventEx( CTRL_BUTTON_START, Data, DataCount, CurrentButtonState[1] & MIDI_GUITAR_BUTTON_START, 128, 0, false );
if( Changes[1] & MIDI_GUITAR_BUTTON_XBOX )
AddEventEx( CTRL_BUTTON_WHAMMY, Data, DataCount, CurrentButtonState[1] & MIDI_GUITAR_BUTTON_XBOX, 16, 0, false );
if( Changes[1] & MIDI_GUITAR_BUTTON_SYNC )
AddEventEx( CTRL_BUTTON_ORANGE, Data, DataCount, CurrentButtonState[1] & MIDI_GUITAR_BUTTON_SYNC, 128, 0, false );
//Set 3
if( Changes[2] & MIDI_GUITAR_BUTTON_TILT )
AddEventEx( CTRL_BUTTON_YGYRO, Data, DataCount, CurrentButtonState[2] & MIDI_GUITAR_BUTTON_TILT, 64, 0, false );

if( Changes[2] & 0x0F )//DPAD
{
//On Events
if( ( CurrentButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_UP )
AddEventEx( CTRL_BUTTON_UP, Data, DataCount, 128, 128, 0, false );
else if( ( CurrentButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_RIGHT )
AddEventEx( CTRL_BUTTON_RIGHT, Data, DataCount, 128, 128, 0, false );
else if( ( CurrentButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_DOWN )
AddEventEx( CTRL_BUTTON_DOWN, Data, DataCount, 128, 128, 0, false );
else if( ( CurrentButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_LEFT )
AddEventEx( CTRL_BUTTON_LEFT, Data, DataCount, 128, 128, 0, false );

//Off Events
if( ( MidiInput[Device].LastButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_UP )
AddEventEx( CTRL_BUTTON_UP, Data, DataCount, 0, 128, 0, false );
else if( ( MidiInput[Device].LastButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_RIGHT )
AddEventEx( CTRL_BUTTON_RIGHT, Data, DataCount, 0, 128, 0, false );
else if( ( MidiInput[Device].LastButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_DOWN )
AddEventEx( CTRL_BUTTON_DOWN, Data, DataCount, 0, 128, 0, false );
else if( ( MidiInput[Device].LastButtonState[2] & 0x0F ) == MIDI_GUITAR_BUTTON_LEFT )
AddEventEx( CTRL_BUTTON_LEFT, Data, DataCount, 0, 128, 0, false );
}

for( int i = 0; i < 4; i++ )
MidiInput[Device].LastButtonState[i] = CurrentButtonState[i];//Update Last States
}
}
return true;
}


In short Method 4 might be what you are looking for.

Thanks
David

[sbeuh]

Hi!

Thx for reply! I had a look at the code and I saw that it expect each string to send midi notes on different midi channels.

switch( InEvent->channel )
{
case 0: Fret = InEvent->note - 64; break;//E High
case 1: Fret = InEvent->note - 59; break;//B
case 2: Fret = InEvent->note - 55; break;//G
case 3: Fret = InEvent->note - 50; break;//D
case 4: Fret = InEvent->note - 45; break;//A
case 5: Fret = InEvent->note - 40; break;//Low E
}

This is how real midi guitar hardware works (One mic for each string and a box that translate frequencies to midi signal). So I assume that if you have a real midi guitar, it will rocks!


As I am using a rocksmith real tone cable and an audio to midi software, there is no string differentiation, every midi notes are send on a single channel. So I always see the high E string strummed.
Currently, I am able to send on my virtual midi channel midi notes with classic "Note On" "Note Off" and "Velocity". "Note On" and "Note Off" contains classical midi notes (ex: 60dec / 3Chex values for a C4).

I assume Phase Shift is currently working with midi values comparison (seeing notes.mid files) but maybe for real guitar part it's a bit more complicated as I see you're sending Change Fret and Strum Events. (I think it's way more simple for keyboard parts)

Could we imagine to add a guitar method which just receive and compare midi notes (like a midi keyboard).

On my little program, I duplicated events for each strings in order to test (ex C4: G string 5th fret was also sending C4: D string 10th fret and A string 15th fret...) And I was able to play an easy song without mistakes even if several notes where sent at a time. (That was the dirty stuff i talked on previous post )

So I think if we can go for a guitar method that call AddEvent(ex?) function with simple midi note. It could allow players to use guitar without all the midi hardware. (Btw bends, slides, and all the stuff will be a bit more complicated :/) I'm also a great rocksmith fan. But their approach seems to be more frequencies oriented.

Well that's a big post here.
Thanks a lot for your time and for sharing this portion of code =)

Have a nice evening!

[raynebc]

It would be cool if Phase Shift would support something like this, but it would have to allow each pitch to be played anywhere on the fretboard because the sound to MIDI logic can't tell on which string each note is being played. The easy way to do this would be to treat each note/chord in the guitar/bass arrangement as a list of pitches that have to be hit, and then see if the incoming MIDI signals indicate they are.

[sbeuh]

Definitely cool!

As I saw by playing a bit with only E string enabled (all notes sent on channel 0) there is already no differences between the string where you hit the note and the result. I was able to play an high E string 5th fret with a B string 10th fret. And it's exactly the same with rocksmith. On some solo (especially on Custom DLC) where I don't like the finger position, I play some of the notes on the upper or downer string and it works well. It seems to be just a raw note recognition.

[raynebc]

David if you were interested in implementing that, I could add data to the MIDI if it would help such as Sysex messages defining the MIDI notes needed for each note/chord in a pro guitar track.

[David]

It seems simple enough to implement myself. The hard part is getting all the gameplay systems to play nice with the change.

[sbeuh]

Sounds really nice!!
thx!

[raynebc]

If the detection accuracy is/becomes reasonably accurate, there could be quite a bit of interest in this from the Rocksmith customs community.