Rainbow Poo, NTSC Synths, and Maker Faire

This last weekend, analog man-about-town Paul Rako asked me to show off some nerd stuff I built at the Atmel booth at Maker Faire New York.

To be totally hashtag-real-talk about it: I was skeptical. I have a lot of complicated feelings about Maker Faire, many of which come out when I’ve been drinking, and which I shall save to share with you, dear reader, until we have a beer together. But. I think Paul rules, and I also owe him big time for letting my unwashed self crash the Analog Aficionados parties AND I also use a ton of Atmel parts for both work and fun AND it seemed like the sort of “networkey personal brand-ey” thing that, while it makes me want to slice myself, is usually a good idea in these troubled and humanistic times.

Here’s the thing I showed off:

Color Me Baddly Analog Video Synth Mark 6

It’s an analog video synthesizer — the sixth prototype I’ve made of one in fact, and this one is finally starting to get OK (you can see some earlier ones if you dig back on the site). It generates NTSC video natively and can spit out either composite or S-Video, most of which looks like so much rainbow flavored wee. But I’m fond of it. The “synthesis” part is basically a processor (an Atmel AVR Atmega, programmed in C and a bit of assembly, at least for now) which generates sync, blanking, and colorburst and a bunch of op amps and transistors which encode control voltages into various video parameters. The majority of the board is taken up by oscillators and signal processors which make or shape incoming waveforms in such a way that they look cool when translated into video.

The synth itself stood up to tons of small children beating on it (some of whom were impressively bright and discovered amazing patches I never would have, and some of whom just wiped peanut butter on the board), I met a bunch of amazing people, and I ended up having a great time despite my initial hey-you-kids-get-that-3d-printer-off-my-lawn attitude. So. THANKS ATMEL and THANKS PAUL you were totally right. And to the grinning hellion with the jelly-hands: I’ve got your number.

The following photos pretty much describe how it went.

MF_Kid
This kid was a genius who made at least two amazing patches that I unceremoniously snatched later. His mom also cracked a joke about chirality.

68Krewgetyrdixout
This is the face I make when I steal intellectual property from children. G-UNIT!

MF_Rainbow_01
Close up of your standard-issue unicorn vomit.

MF_Coffee
Also, this happened.

Big Ups to Adam and Lexie for taking such great photos and providing stellar moral support.

Vectors, or “My Man Inf Left A VEC and a 9 At My Crib”

So awhile back my man Andrew Reitano dragged home a vector monitor from an old Asteroids arcade cabinet.
Here’s what we’ve done with it so far:

It was a long road to get there!
For those of you born after the cold war, Asteroids was a hugely popular game by Atari which used a now-obsolete style of monitor. If you haven’t played one in an arcade you owe it to yourself to do it, because they look like nothing else. The phosphor artifacts are trippy and beautiful.

Anyway, the trick with these things is that they get driven more like an oscilloscope than like a normal TV or raster monitor. Meaning, you send them voltages and deflect an electron gun to a certain spot on the screen, rather than drawing an image out line by line like an NTSC or RGB display. This presents all kinds of problems.

First, you have to move this electron gun at an even rate, and you need to do it on a halfway stable and reasonably fast way. I think Asteroids drew at 4MHz, meaning you needed to be pushing the gun that often AND slewing a beam that fast. After blowing up a handful of output transistors in the monitor, we came up with these:

VEC9_FPGA_and_Amp_small

That’s a Xilinx Spartan-3E FPGA on a Nexys2 dev board which Andrew programmed to be our vector generator, and a DAC/Amplifier board I designed to sling the beam around at 20 MHz and 12-bit resolution. First thing we did (this was with some gnarly R2R dac) was get a cube to display on the scope (I think the monitor was still fried from earlier “tests”). It kinda sucked:

Vec_test_05_small

Encouraged, we got drunk, fixed the monitor, Andrew wrote a rasterizer and we put the patron saint of Bed Stuy on the screen. It sucked less:

Vec_test_02_small

By then we were pretty sure we had a new arcade game in the works and the heavy lifting started. Andrew tuned the hell out of the VHDL and got the FPGA tweaking all kinds of gun parameters. I fixed the amp a couple times to get it running that gun right, and both of us spent a lot of time coding in C. I’ve wanted to make video games ever since I was a kid, but somehow this was my first foray into coding 3d rotations, particles, dynamic memory, and a lot of other generally non-embedded stuff. It ruled. We got to the point where we could rudely waste 3d polygons:

VEC9_Screenshot_Jul25_small

We wrote a storyboard about the Soviet Union blowing up Chicago, threw in a bangin track by Nicholas “Windbreaker” Read and coded a chaingun. Lastly, one awesome thing is that since we’d basically made a vector “video card”, it was not too hard to generate a commlink / stores display / HUD using a monochrome VGA display, running as a secondary monitor. It looks amazing!

We’re pretty sure we’re on our way to a totally kickass arcade cabinet once we get the real game coded. If you know what we should do with this hot mess once we’re done, hit us up!
TB, August 2013

Tektronix 564B Scope Repair, Part 1

So, I have a real thing for old Tek scopes.

This is due in no small part to the gospel preached by Jim Williams (you can get the cliff notes for said polemic here as well as a lot of other great scope-related stuff and some repair notes).

More personally, it’s also due to the fact that my first scope was a Tek — a 454 that I snagged from trash pile at my college job before I knew an opamp from an opcode. I learned how to use a scope by fiddling with that thing, and I broke it a lot being a careless kid (mostly in shipping). This meant I had to fix it a lot, too. My first job out of school was repairing stereo receivers and guitar amps, and fixing that 454 was both alien and awesome. Alien, because it was way more complicated than a Fender Twin, and awesome because the service manual was head and shoulders above any repair treatise I’d ever seen, whether it was for a piece of electronics, a car, or a piece of software. It really wanted you to understand the instrument. I loved that manual.

So later, once I started drinking the Williams kool-aid, it didn’t take a whole lot of convincing that the old Teks were “intellectual integrity” reified. I won’t bore you all with the details — Jim did it better anyway (as does Kent at his excellent site).

Anyhow, I bought an old Tek564b off Ebay a long time ago and had it sitting in the project pile. I got it because I was fascinated by the idea of the Analog Storage Oscilloscope. In a nutshell, an ASO allows you to save a trace on the screen of an oscilloscope by using some special phosphors in the CRT and a set of special electron guns. You can save any number of traces (two is easy, any more and it gets tricky) to compare or photograph. Since the stored output is not sampled, it essentially has infinite “bit-depth”, and it was built at a time when semiconductor memory was still on the drawing board.

This particular 564 and its plugins had a few problems, but one really stood out as a great example of the interrelated-ness of systems inside electronic equipment. I made the following video detailing this problem before I fixed the scope. At the time, I didn’t know what was wrong with the scope but I had a pretty good idea. I’ll give you a hint — if you are thinking along the same lines as I was at the end of that video, you’re wrong :-)

In the coming weeks I’ll post the actual repair, but play along at home and see if you can figure it out.

BLDCs, the Wrong Way

Hall Sensor Boo Boo
The above is a set of scope traces from the three coils of a BLDC which is being commutated incorrectly. Read on.

The job that’s been paying my bills and keeping me away from artsy-fartsy circuits for the past six months involves making a set of these enormous robot doors for a Certain Very Fancy Person’s house. Each door is 13 feet tall, around 7 feet wide, and weighs 1500 pounds. There are 66 of them in said house, and more in the servant quarters(!?!). The circuits on board each door have to handle running an onboard air compressor (which regulates a pneumatic weatherseal) as well as keeping track of temperature to linearize the pressure sensors when the weather gets cold. They also have to charge and maintain sealed lead acid batteries. They have commutated power rails. They have to communicate over said power rails, and do so using an capacitively-coupled data slicer and a proprietary protocol I wrote. This protocol has to be robust enough to bootload the processor over. It’s a proper embedded systems job.

All this having been said, the great majority of the real estate on the PCB is dedicated to running the big BLDCs that make the doors move. This post is some observations particulars of said motors.

I’d run lots of standard DC motors in past, and built H-bridges galore, but this was my first encounter with BLDC driving. As such, I made some mistakes in the prototype. And I am going to lay those out for you, dear reader, such that you might learn from the fires of MY workbench.

The motors in question were Maxon EC-Max 30 series motors with a gearhead. They are small, powerful, and expensive. We decided to use BLDCs for this job primarily because of their size, but the fact that the reliability is generally greater (no brushes) than a standard brushed motor certainly was a bonus. The BLDC driver circuit needed to fit into the stile of this door, which was only 1.5″ wide or so. The motors could be expected to stall under certain conditions, so the motor drivers had to be more Conan than not.

Once that driver was designed and fabbed, the fun on the workbench began.

A quick BLDC review:
Brushless DC Motors are like normal DC motors — except the user has to handle commutating the motor on their own. This means, more or less, wrangling the electric fields in the motor at the right time such that the motor turns. A standard DC motor has a set of “brushes” (usually pieces of carbon) that squeeze against the moving part of the motor (the rotor). These are responsible for getting power from stationary part into the spinning part. As the rotor turns, the brush presses on the correct bit of the rotor to get the fields doing their thing. This is cool because it’s easy to drive and relatively simple mechanically. The BAD thing about it is that the graphite/carbon brush wears down over time, AND the brush-scrape nature of the contact makes a world of tiny electrical arcs inside the motor which look cool but kick off a ton of electrical noise (and audio noise, which in this case mattered).
In a BLDC, it’s YOUR job to energize a set of coils at the right time to keep the motor turning. There are three (usually) coils in the stationary part of a BLDC and a number of magnets in the rotor. The trick is to energize the right combination of coils such that you pull the right magnet to the right spot at the right time. If you do this, the motor turns smoothly. It’s a sort-of-not-really like timing a spark plug, if you’ve ever had to do that.
The trick is, you need to know WHERE the rotor is at any given time, such that you can know when to turn off the last set of coils and turn on the next set. There are lots of ways to do this, but a common one is with hall sensors which read the rotation of the motor shaft via some magnets on that shaft. These are usually conveniently built into the motor. In my case, they read with 60 degree resolution, which is plenty to keep the motor moving correctly. Often, either a piece of hardware or an ISR reads the hall sensors, and then energizes the correct coils. This whole process can be as simple as a look up table.

There are lots of documents on the internet about BLDCs, but there are relatively few about what happens with incorrect commutation. The interesting part is when you mess it up and try to figure out what went wrong.

First step, the datasheet: The Maxon datasheet more or less recommends that you buy the Maxon motor drivers and just be done with it. Our form factor, target budget, and my not-insignificant sense of pride ruled that out. It DOES have a commutation chart, but it’s pictorial and is not that clear about “active high / active low” and other minor details like that. So, I made some guesses and let her rip.

In the course of letting her rip, I saw all kinds of exciting (read: trainwreck) motor beahvior. I figured out how that Maxon ran, but I also made a handy set of rules of thumb about what BLDCs do when you drive them wrong. From this, I made some procedures with which I ought to be able to start out with any unknown three-phase BLDC with hall sensors and derive how to get the correct drive sequencing. It’s like this:

Observations:

1.) BLDCs _WILL_ turn with some combinations of incorrect drive. This can mislead the un-wary.

2.) BLDCs usually not turn if the commutation sequence is mixed up. I assumed this would always be the case. It isn’t.
When a BLDC has a commutation sequence which is out of order, usually you see it STOP at some position and lock up, drawing stall current. If you aren’t on a current limited supply, or your motor drivers are not like Conan, you may damage your drive section. This is because whatever coil you’ve energized is pulling the BLDC to the wrong place and then KEEPING it there, since another set of halls has not been triggered. This probably means you have your driving steps out of order (or are energizing the wrong combination of coils).

3.) The interesting part — the BLDC _WILL_ spin, maybe, if you have the commutation sequence right BUT offset from the hall sensors readings. Meaning, if you are always energizing your coils one step ahead of where they should be, the BLDC will pull the motor to the right spot, but it will draw a lot of power doing it AND you will not have very much torque. If this happens, you probably have your hall sequence right AND your commutation sequence right, BUT they are lined up incorrectly.

Corollary:

1.) If you aren’t sure what your hall sequence is, you can spin the motor with your fingers (or a wrench, or whatever) and read the correct order of hall transitions.
2.) If you aren’t sure what your commutation sequence is, you can try messing with the order of different valid coil combinations and running your motor OPEN LOOP. This means stepping through your coil combinations on a timer, energizing them, and seeing if the motor turns. It will turn with bad efficiency again, and probably draw a lot of current, but it will turn when you have the sequence right. Best to do this on a current limited supply.
3.) Once you have those two pieces of information, you can line them up and figure out the correct drive sequence. This sequence will be the one in which the motor spins with the greatest torque and the best efficiency in both directions (IE, it has to work when reversed, too).

An unexpected benefit of doing this is that you see all kinds of BLDC commutation problems. This is useful when you’re debugging a system later, and have to deal with broken wires or damaged drivers, and can guess symptoms much more easily.

Component Variation, Or, The Least Sexy Electronics Problem Evar

Analog is sexy, we all agree, right? Embedded systems on the other hand, are full of lots of unglamorous problems. Filesystems, say. Inherently un-sexy.

Yawnz

But I think component variation is maybe the best, most un-sexy problem there ever was. The unsexy cherry on the diet sundae. Like, you HAVE to solve it if you are making lots of something or that thing as a population will suck, even though the one on your bench always ruled.

All the pots in WTPA2 are these custom Taiwan Alpha jobbies. There are two values, 10kA and 100kA. The VCO uses one of the 10kAs as a coarse control, and it sets the voltage into a current sink which in turn sets the frequency. I’d been messing with the op amps in this circuit to try and get some performance improvements and “all of a sudden” one of the DUTs didn’t work correctly. At first I figured it was the opamp change, but after a lot of measurement and desoldering and component testing, it turned out one of the 10k pots was really 11.4k. This was a greater than 10% variation!

I’d built a margin in for error, but this was above it, and the current sink was getting too high of a voltage. I tested a dozen pots or so from the bin, and all of them were much less off. Still, since one was off, probably another one could be as well. It could even have been a result of the soldering process. I actually bothered to do a DC simulation at this point (using qucs) and fiddled with the component values until they were all as off as I could imagine them possibly being, and then resized the scaling resistor that sets the upper range of the VCO. It was a really crappy annoying unsatisfying solution, because it means that MOST of the units will be operating at a slower maximum clock than they need to. But that one in twelve or one in 100 will work correctly. Serves me right for getting the cheap pots, but there you go. Margin. Component variation.

Least Sexy Problem Evar.

TB