0

I want to implement a simple musical keyboard with Arduino using tone() and noTone() functions, etc.

The desired behavior is:

  • While PC and Arduino are connected via USB/Serial...
  • ...when I press a key, a corresponding note starts playing...
  • ...and if I release that key that note stops playing...
  • ...or if I press another key, another note replaces the former one.

So, basically, instead of "while key is pressed" logic, I would like to just write a byte to serial when a key is pressed, write nothing while the key is held pressed, and write another byte when that key is released.

I will be using a Python script in the PC, but examples in other languages, or just using a plain serial monitor, are fine.

1
  • PCs' keyboards send key-down and key-up signals to the PC so that much is available already. But that gets translated into letters including generating key-repeats at a fairly low level - BIOS, I think. Getting that deep from a higher level language like Python may be a challenge. A search for "windows keydown" returned a few hits that looked promising.
    – JRobert
    Commented May 23, 2015 at 15:37

4 Answers 4

1

As mentioned by Spiked3 there are a few ways to hit the ground running with C#.

Firstly, as detailed by Majenko, you will need an application running on your PC and some code on the Arduino to interact with this. For the desktop application, the two easiest ways to get started with C# are with WinForms, or XNA/MonoGame. WinForms is targeted at Desktop application development; think legacy desktop apps. Although it is not the current defacto standard (replaced largely by WPF) it is very simple and akin to Swing in Java to use. MonoGame and XNA are targeted at video game development. The latter would be my preference as it lends itself more towards real time applications, and has a game loop much like the Arduino so it is very easy to start using. The caveat is that there is more setup involved, but you will probably find this the friendliest and best supported.

Firstly, The Code! So for the Winforms Route, We have something like this:

using System.Collections.Generic;
using System.Diagnostics;
using System.Windows.Forms;

namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void Form1_KeyUp(object sender, KeyEventArgs e)
        {
            Debug.WriteLine(" Key Up! ({0})", e.KeyCode);

            // TODO: Talk with Arduino
        }

        private void Form1_KeyPress(object sender, KeyPressEventArgs e)
        {
            Debug.WriteLine(" Key Press! ({0})", e.KeyChar);

            // TODO: Talk With Arduino
        }
    }
}

To get to this point, fire up a new winforms project, and open up the 'Form1' code (the default window which is added to the project) add the methods as outlined above. You will need to go to the designer afterwards and add events for on KeyPressand on KeyUp to tie these functions to the Window. You can do this by clicking the Window, then opening the Properties Panel, and then clicking the lightening bolt. You will see a list of event names; find the corresponding events (Key Press and Key Up) and associate them with these methods.

From here, you can perform some logic when a button is either pressed or released.

The monogame or XNA approach is a bit simpler to follow as there is no hidden magic; but may be more intimidating if you're not familiar with C# and the project type.

using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Input;
using System.Diagnostics;

namespace GameName1
{
    public class Game1 : Game
    {
        GraphicsDeviceManager graphics;

        KeyboardState currentState;
        KeyboardState previousState;

        public Game1() : base()
        {
            graphics = new GraphicsDeviceManager(this);
            Content.RootDirectory = "Content";
        }

        protected override void Initialize()
        {
            // Get state management started
            currentState = Keyboard.GetState();

            base.Initialize();
        }

        protected override void Update(GameTime gameTime)
        {
            // Listen for exit command
            if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed || Keyboard.GetState().IsKeyDown(Keys.Escape))
                Exit();

            // Update the state of your keyboard object
            previousState = currentState;
            currentState = Keyboard.GetState();

            // Detect Keypress of key 'F'
            if (currentState.IsKeyDown(Keys.F) && previousState.IsKeyUp(Keys.F))
            {
                // F Key has been pressed
                Debug.WriteLine(" Key Pressed! ({0})", 'F');
                //TODO Talk to Arduino
            }

            // Detect Keyrelease of key 'F'
            if (currentState.IsKeyUp(Keys.F) && previousState.IsKeyDown(Keys.F))
            {
                // F Key has been pressed
                Debug.WriteLine(" Key Released! ({0})", 'F');
                //TODO Talk to Arduino
            }

            base.Update(gameTime);
        }
    }
}

In either case, you will need to get yourself set up first. Get a compy of Visual Studio Community from the link below, and install MonoGame if desired. Then do some C# tutorials to get comfortable in your new environment.

Lastly, you will want to talk with your Arduino. This can be done by implementing a compatible Serial interface between your desktop application and your Arduino. See the tutorial below to get started!

I hope this information is useful. There is a very broad range of topics mentioned here, and it may seem overwhelming at first - but C# is a very idiosyncratic language in that it permits rapid development in many different scenarios, and I think that you will find it rewarding to conquer the learning curve.

5
  • Incidentally, my current job is C#/WPF developer, so creating a GUI app is something I am quite familiar with. The greatest "paradigm change" here was to realize that everything I wanted to do regarding "capturing keyboard events" is already implemented in any GUI toolkit (I ended up using PyGame), while most examples use command-line tools and/or serial monitors on a terminal. Commented May 25, 2015 at 14:10
  • Yeah I do WPF too (spiked3.com). As far as I know SlimDX has more or less replaced XNA for current development. In any case keyboard events are native to all windows development environments, except raw python, that I know of. Why would you not just use WPF?
    – user6569
    Commented May 25, 2015 at 18:00
  • Don't get me wrong - I love WPF, and strongly dislike WinForms in contrast, however I generally find myself strictly following the MVVM design pattern with WPF and consequently I find it difficult to get in that 'single dirty file with a blocking UI thread' mindset. :)
    – Élie
    Commented May 26, 2015 at 1:43
  • I just took a quick look at SlimDX - it appears that it more or less just wraps DirectX, which isn't a bad thing. I think MonoGame and XNA will still be simpler for most use cases where you would use managed code, as there is a lot of automation that I am sceptical would be included with SlimDX, especially the content pipeline. I get the impression SlimDX is related to SharpDX, and that it's best equivalent to XNA would be MXA. mxa.codeplex.com
    – Élie
    Commented May 26, 2015 at 1:49
  • 1
    Mxa says it is also based on sharpDX, and it just doesnt have the volume yet of slimDX. I'd love to continue the conversation, but not the right place. Email me spiked3 @ gmail com if you want to talk more.
    – user6569
    Commented May 26, 2015 at 14:31
0

You need to come up with a protocol. Then you need to write the PC software. The PC software will monitor the keyboard, and when you press a key it will send a certain signal through the serial port. When you release the key it will send a different signal.

How you arrange that is entirely up to you.

You might implement a protocol that sends something like:

b4
c4
d3
off
c3
f3
off
g4
off

Where the first letter is the note, the second letter is the octave to play that note in, and the word "off" is to stop playing.

I don't have a clue (and probably few people here do) how you'd program the PC end in Python. It most likely depends on what platform you're working on as to what kind of interface you can build.

0

It sounds really simple, but as I think about it, the python twist makes it near impossible.

First the simple part. What you are describing is very close to the MIDI spec. It define a sequence of character for note on/note off events over serial. I'm rusty but it something like this x'90' + chan1..16, note, velocity (3 bytes for note on) and x'80' + chan1..16, note, velocity (3 bytes for note off). (spec)

Firmata took that idea, and turned it into a remote arduino protocol, where instead of a note, a pin number is used (eg pin on/pin off) and you could dig into the firmata that is included with Arduino for some ideas.

The hard part is the keyboard. I use a function getch, that I define myself based on whether the script is running on a POSIX host or not (although that is technically inaccurate). It is described at https://stackoverflow.com/questions/510357/python-read-a-single-character-from-the-user but unfortunately it only waits for a keystroke and provide no information about a key release. You are going to have to dig deep and probably write some platform specific code, maybe even a C .DLL, to get that kind of interaction in python. Handling key down/up is trivial in any language but python (for windows anyhow, dont know about *nix)

8
  • 2
    Simplest would be to use a GUI toolkit to make a GUI application. GTK+ would do the job - anything that can give you keypress / keyrelease events.
    – Majenko
    Commented May 23, 2015 at 19:27
  • yeah, that is why I said it is trivial in anything but python. Not sure how (or even why) it works with python GTK.
    – user6569
    Commented May 23, 2015 at 21:26
  • @Spiked3 when you say "anything" but python, could you suggest something to use instead? I am using python just because it's what I am most familiar with, but it's not fundamental at all to use strictly python. Commented May 23, 2015 at 21:28
  • C# would be (is) my primary choice. I'm looking for a piano keyboard gtk app that might run with python but have not found any yet. I think even Processing probably could do it but I am not sure. It is probably worth looking at though, since it is so similar to Arduino (IDE)
    – user6569
    Commented May 23, 2015 at 21:32
  • 1
    processing.org/reference/keyPressed_.html
    – user6569
    Commented May 23, 2015 at 21:33
0

UPDATE: When I posted this answer I had not yet any knowledge of the Processing Language. If you came to this answer with the same intention I had (the question title), I strongly suggest you to use Processing instead, which has much more straightforward keyboard event capturing and serial communication working out of the box with minimal code.

A question showing this in action is here: https://stackoverflow.com/questions/18366482/processing-to-arduino-with-keyboard


After realizing that what I actually wanted was to "capture keyboard events", I ended up using PyGame due to its minimalistic setup. Now I can send whatever I want to the Arduino using PySerial.

PS.: Don't be fooled by the lame aesthetics of PyGame website, it is a very mature gaming platform.

Full code here: https://github.com/heltonbiker/Experiments/tree/master/SerialMusicKeyboard

#/usr/bin/env python

import pygame
import serial
import glob
import time

def pygameSetup():
    pygame.init()
    screen = pygame.display.set_mode((468, 60))

    background = pygame.Surface(screen.get_size())
    background = background.convert()
    background.fill((250, 250, 250))

    if pygame.font:
        font = pygame.font.Font(None, 36)
        text = font.render("Shut up 'n play yer keyboard!", 1, (10, 10, 10))
        textpos = text.get_rect(centerx=background.get_width()/2)
        background.blit(text, textpos)

    screen.blit(background, (0, 0))
    pygame.display.flip()


def main():

    pygameSetup()

    initialized = False

    while not initialized:
        try:
            serialpath = glob.glob('/dev/ttyACM*')[0]
            ser = serial.Serial(serialpath, 9600)
            initialized = True
            print "serial up"
        except:
            time.sleep(500)
            print "waiting for serial"

    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return
            elif event.type == pygame.KEYDOWN:
                print event
                ser.write(chr(event.key))            
            elif event.type == pygame.KEYUP:
                ser.write(chr(0))


if __name__ == '__main__': 
    main()
1
  • @heltonbiker consider accepting any of the answers as "the answer" as currently, the question seems "unanswered". While you actually have found something usefull. By accepting an answer, no-one with the intention of giving an answer will waste time on this question.
    – aaa
    Commented May 26, 2015 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.