Australian Code Breakers

On Wednesday evening I went along to a talk at a nearby library by David Dufty about his recent book ‘The Secret Code-Breakers of Central Bureau – How Australia’s signals intelligence network helped win the Pacific War’ published last year by Scribe.

The Secret Code-Breakers of Central Bureau by David Dufty

It’s a great story that does uncover previously unacknowledged contributions. Dufty’s interest was sparked by a newspaper mention of Australian wartime code-breaking on Anzac Day 2012. His interest triggered a comprehensive research trail.

It’s a great read with a solid bibliography. He interviewed about twenty people who worked on breaking the Japanese codes. From a standing start, the operation grew to involve over 4,300 Australians – a venture, Dufty says,  it’s hard to imagine us being able to mount as readily today.

He mentioned many of the characters from Australia’s early radio history, including Mrs Mac, Florence MacKenzie, who trained thousands of women morse operators who in turn were used to train many Australian servicemen.

He also mentioned Eric Nave who was responsible for breaking Japan’s Naval codes. Nave as a young naval cadet had spent years in Japan learning the language and culture of the country.

The character with the best nickname would have to be Keith ‘Zero’ Falconer. He was the country’s top interceptor of Japanese Kana coded messages. He got the nickname from his colleagues as every single day of Kana code training in Melbourne he would score zero errors in the test. Japanese hams can still be heard conversing in this code on the bands today.

Japanese Kana or Wabun morse code
Japanese Kana or Wabun morse code

The character who stands out from David Dufty’s talk on Wednesday evening is Stan or Pappy Clark. Apparently, prior to enlistment, his work was scripting radio serials for children. The mention of the magic word radio was enough to catch the eye of people recruiting for radio intelligence work, and it turned out to be a fortunate selection for Australia.

Stan Clark used his talents to develop a comprehensive knowledge of the Japanese communication networks and was able to analyse the dynamic ebb and flow of their radio traffic. Even if we weren’t able to decode every message the broad overview – which Dufty interpreted as the ‘metadata’ of the enemy’s radio communication – of this traffic analysis played a crucial role in determining the allied strategy of the war and effectively saved thousands of allied and enemy lives. Macarthur’s famous island hopping strategy was directly informed by this intelligence. One of the special things about Wednesday night was that unknown to Dufty until the end of his talk, Clark’s grandson and family were in the audience.

Learning morse and touch typing in tandem

I’ve been wondering whether I should try to synchronise my most recent efforts at learning and improving my morse with a similar complementary neural mapping exercise of simultaneously learning to touch type as I practice copying morse code.

I was googling around – on the off-chance someone had developed the ultimate piece of software which combined G4FON Koch CW Trainer with Typist or some other touch type trainer, oh and for the Mac would be good – using the term “learning morse and to touch type at same time” and I discovered via Google Books Lewis Coe’s ‘The Telegraph’. Here on page 109 is mention of how operators could recognise their call in their sleep!

 

The highlighting is due to the google search term.

On pages 69-70 there’s mention of how the older operators used a mill to take down messages as they streamed in over the line.

Maybe the best approach would be to change the learning sequence of characters to match the character sequence of touch typing so that every character gets a double whammy of learning reinforcement. So F, J, D, K, S, L, A and ; (?) instead of K, M, R, S, U, A, P, T, L and O. 
It shouldn’t be too hard to generate suitable static mp3 audio files of the touch type progression of characters. It would be great if there was a way to randomly generate according to this new progression, in a similar way to the G4FON software, with the alternate character sequence.
Also, I’m sure someone somewhere must have considered the learning pros and cons of such an approach.
I also found the perfect font to use. It’s called MILL
Testing the MILL font
You can grab this TrueType font from http://www.qsl.net/n1ea/MILL.ttf.
Not quite the same as this one which explains its own provenance:

Joe Taylor K1JT presentation on WSJT-X

Joe Taylor, K1JT gave a 75 minute long presentation on WSJT-X at the recent MicroHams Digital Conference in Redmond Washington in mid March.

This link takes you to the section within the full day’s 8 hours and 21 minutes long recording.

This link takes you to a version edited by Budd, WB7FHC of the same talk.

During the presentation, Joe Taylor showed this slide indicating the comparable signal to noise limits resolving different weak signal modes compared to traditional modes like SSB and CW.

K3 and digital modes

Using WSJT-X as my example digital program, here are the steps to getting digital modes to work with the K3 using an external sound card, the ASUS U7.
The process is almost identical to the one I use with the KX3, an external soundcard dongle and a MacBook Pro.
First, it is simple and easy to adjust and repeatable.
Connections:
Line in on the K3 rear panel is connected to the Headphones socket on the front of the Asus U7 (using red patch cord with ‘Line in’ label).
Line out on the K3 rear panel is connected to the Microphone socket on the front of the Asus U7 (using black patch cord with ‘Line out’ label).
RS232 on rear of K3 is connected via USB adaptor to unused USB connector on the PC.
USB socket on rear of Asus U7 is connected to another USB connector on PC. (this also provides power for the Asus U7).
software:
Asus (Xonar U7 Audio Center) to confirm selection of Headphones and Line-In, and to adjust levels
WSJT-X File > Settings or F2 >
General tab: Enter your callsign and grid square
Radio tab: Select Elecraft K3/KX3 in Rig drop-down list
Set serial port to same COM Port number revealed when you are using
Elecraft K3 Utility (But both programs cannot be run at same time)
Set baud rate to 38400
Data bits 8     Stop bits One    Handshake  None
PTT  – select CAT     Port should be USB
Transmit Audio Source is Rear/Data
Mode is Data/Pkt
Split Operation is set to Rig
Use Test CAT and Test PTT button to confirm correct connections. CAT will turn Green and PTT Red after clicking, indicating all is well.
Audio tab:
Under Soundcard select the appropriate option from the drop down lists:
Input – ‘Line (2-Xonar U7)’
Output – ‘Headphones (2-Xonar U7)’
Under the Reporting tab you can enable or disable uploading of spots to PSK Reporter and linkages to other programs include logging software.
Under the Frequencies tab – if you find there are no frequencies listed for the different modes, position cursor in the Working Frequencies window and right click and select ‘Reset’. This should populate the window with all the frequencies.

Use the PC’s regular audio device controls or the soundcard’s controls to adjust input and output levels. For example WSJT-X likes to have the green input bar graph on the lower left of the application window indicating about 30dB. Adjust the Headphones level to achieve this. I operate routinely with it hovering between 30 and 60dB.
The K3 is particular about the level of ALC on its transmit signal. You can use the microphone level to easily achieve the Elecraft ideal of 4 solid bars with the fifth bar flickering. This ensures a clean splatter-free signal.

Other aspects you need to consider with applications such as WSJT-X include installing an application to keep the PC clock accurate, as well as eventually checking frequency alignment.

The book I’d like written/to write about FT8, WSPR & other digital modes

I’ve spent the last couple of weeks spending some continuous and focused time finally getting some consistent results out of my radios using digital modes, especially WSPR and JT8 using WSJT-X.
Like everything else I do with ham radio, they reveal the woeful inadequacy of my antennas here. In fact, one of the benefits of WSPR is, in fact, the information it yields that provides some useful and comparable data about your antenna performance.  The great thing about WSPR is that the worst antenna still seems to manage to generate some data so that any subsequent ‘improvements’ can be evaluated.
I have yet to find a document that helps you understand what the numbers especially the SNR actually mean. If you hear a signal at 0dB via WSPR does that imply that a CW signal at the same power output or some number of dB power increase would also be able to be copied? These kinds of questions are what leads me to think that there would be high interest in this kind of information. Even if there isn’t that potential market, it’s still an idea worth pursuing out of personal interest. I’d also like to read an explanation of what the variation of SNR numbers from the same station indicate about changing ionospheric conditions etc – what is significant, and what is within the range of normal statistical variation.
I’d love to know how to really make use of the data gathered by WSPRnet. What techniques do people use to manipulate and analyse the data? Also, there are presumably assumptions that need to be tested or acknowledged. Many people running a WSPR beacon leave the radio and antenna untouched – so it’s a constant – but others might explicitly be using WSPR to run tests of new antenna equipment so that the antenna is changing and not a stable element. I have been using it for exactly this over the last few days and noticed a significant increase in the number of reports and the quality of the signals reported in response to an extra metre in height of an antenna over the comparable time of day. But maybe the changes are within the range of normal day-to-day variation – especially at the current low phase of the sunspot cycle.
But I’m confident that a consistent user would be able to make some pretty reliable assumptions based on extended observations about what beacons could be considered constant if only from their numbers over the months. It would be great if there were some functions developed that could be accessed online on the WSPRnet site to analyse numerical qualities and features of the more consistent and reliable beacon stations. It would require some computer grunt I suppose to host these server-side processes which might be beyond the budget of the current setup. Whenever the number of concurrent users exceeds 120-140 the WSPRnet site regularly seems to crash and takes quite some time to recover.
It would be great to gather info on the different ways people use WSPR data. How much do professional space weather researchers use the immense volumes of data now being generated? I understand that Australia’s Space Weather Services staff do use it. As I write this WSPRnet announces it has counted over 952 million spots and is adding over a million every day!
I would love to see how people have used data gleaned via WSPR to generate views of how propagation changes during the day for different bands, or how to synthesise the same data into an informative comparative analysis of antenna systems.
Sotabeams DXplorer system – sold as part of the WSPRlite package – performs some interesting analysis of the WSPR data to generate logarithmically(?) scaled graphs of the distance of WSPR reports. It would better if somehow the formula underlying any comparison was able to also take into account the different power levels used.
I would also like to have a concise explanation of frequency calibration. The material K1JT includes was authored almost a decade ago. I’m unsure whether the latest versions of the application accommodate these procedures. I suspect they do as there is a need to do it. I am noticing a deviation in my reported frequency cored to what I calculate it should be. It’s quite out of the ballpark. Similarly, the variation amongst ‘reporters’ is relatively wide. Simple and direct advice about how to deal with this and fix it would be great.
There’s probably still some room to think about more ways to exploit the potential of the massive group of users for various types of experiments investigating propagation etc. Using the different modes it’s amazing how they can illustrate the variety of space weather behaviour. Earlier tonight while using FT8 on 20 metres, I saw the band suddenly go blank. It was as if the antenna had been disconnected. It recovered a little later.
Also, different ways of using the applications – especially WSPR – can lead to quite different impressions about what is actually going on. Running WSPR in band hopping mode in late evening creates the impression that there is no activity at all across those bands. However locking the system onto one band – 40 metres here – reveals a deeper level of ongoing activity that the thinner sampling of the band hopping mode misses representing.
At a more nitty-gritty level, I would like to know how best to use the different modes. Gary Hinson G4IFB/ZL2IFB’s FT8 Operating Guide is very clear. The main WSJT-X User Guide seems to me to be more about the program rather than practical information about using it. It seems to assumes a high level of background and technical familiarity. I suppose I’d prefer documentation that describes explicitly and in a good level of practical detail how people are using the applications. I think it’s brilliant that the WSJT-X application is available for MacOS and Linux as well as Windows and even for the Raspberry Pi!
I wonder how far away we are from radios – kit or commercially produced – that are designed just for FT8 and nothing else. I think Adam Rong – a seller of QRP kits and radios developed and built in China – is about to offer such a radio.
As someone who has spent an inordinate amount of time *not* managing to get digital modes to run on my radios for a long time, I am delighted that in the last few weeks I have been able to get three of my Elecraft radios to successfully decode. It all seems so simple to me now, so that I can’t understand the barrier was before. Well, I think it may have been what I’m writing about here – the lack of suitably written documentation. What got me off on a successful run was a very simple PDF file about running sound card digimodes on the KX2 using a cheap soundcard dongle. It also took into account the Macintosh – so my first success was using the KX2 and my MacBook Pro. It was totally simple and easy to migrate the whole approach across to a KX3 when I wanted to run that much more power. Watching the PA temperature on the KX2 rise during a WSPR transmission – getting up to 53 degrees C on 2 watts – was exciting. I then substituted the Asus Sonar U7 sound card in place of the dongle. And then when I wanted to crank up the power a bit more for FT8, I migrated over to the K3. It was very satisfying to be able to adjust the Power slider in the WSPR mode to get the four solid bars of ALC with the flickering fifth. Also to get a clear sense of the actual level of the signal in that was required to get the decodes to start appearing.
At the moment I’ve been snatching data from the WSPRnet reports, taking a snapshot of the map view of my transmissions over say a day, and also taking snapshots of HamCAP each hour of the day’s propagation predictions for that band.

Magnetic Loop – step by step

Well, the replacement stepper controller chips arrived early this morning. Didn’t take too long to replace them and connect everything up for a test.

The good news is that after a slight adjustment of VR2 the stepper appeared to be controlled by the rotary encoder. It made the right noises.

Try as I may, I wasn’t able to carry out the initial setup procedure as outlined by Loftur. I couldn’t find a peak in the noise.

After applying a portable SWR analyser directly to the loop when I thought I was at the lowest frequency point the loop would tune, I discovered there was a dip around 23 MHz! I was nearer the other extreme of what I think this loop’s range should be. So I’m thinking I may have the stepper motors wired wrongly with the result that the stepper goes in the opposite direction to what the controller thinks it should be.

After switching one pair of wires at the controller end, the stepper made a terrible noise as it hit the end of the capacitor’s travel. Checking it tentatively it didn’t seem to distinguish between clockwise and counterclockwise movement of the encoder. Whatever you did, the stepper turned in the same direction.

Time for further closer inspection of the wiring around the newly re-oriented common mode chokes and their associated bypass capacitors. This kind of behaviour may have something to do with a missing or failed connection in this part of the circuit.

Stepper only appears to turn CCW which turns the VVC CW. I’m now hoping the glitch is pretty obvious in this part of the board or associated wiring.

Magnetic loop progress

Finally found some time to re-orientate the two transformers. I decided to remove the eight bypass 10nF capacitors to give my soldering iron more wriggle room to remove the transformers. The solder on the transformer pads needed some encouragement – in the form of more molten solder – to loosen up. This allowed me to use solder wick to take away most of what was holding them in place. The corners of the transformer are pretty fragile and would not tolerate much heat. Wasn’t too hard to lift them. Much more complicated was cleaning up the holes where the eight capacitors had been. I managed to destroy one of the pads. Decided to stop destroying the PCB and to re-install the bypass capacitors on the other side of the board.

I was actually able to use most of the capacitors I had removed and checked every connection a couple of times at least, especially a couple of the tiny pads for the caps that are near the larger choke pads.

Soldered the transformers back in properly this time, again checking every step over and over again.

After re-orienting the common mode chokes...
After re-orienting the common mode chokes…
...and replacing the caps on bottom of the PCB to avoid damaging fragile pads.
…and replacing the caps on bottom of the PCB to avoid damaging fragile pads.

Taking a few deep breaths now and having a break before reassembling the whole device and connecting it to the radio and the loop for the moment of truth and to find out if the A4975 stepper driver ICs can come back to life after having both their two outputs accidentally connected together! Normally they are connected across one of the stepper motor windings.

——

Well, because there is still no response from the stepper motor, the answer, unfortunately, is no, they probably need replacement. At least this time they’re not red hot. I have a pair of A4975 chips due to arrive sometime on Monday. Essentially $4.90 a pop, no shipping charge.

And reading the datasheet on the A4975 I see on page 10 mention of the thermal protection circuitry that shuts off the output transistors when the junction temperature reaches +165 degrees C. “This is intended only to protect the device from failures due to excessive junction temperatures and should not imply that output short circuits are permitted.” And I reckon the tiny choke winding is close enough to a short circuit.

At least we are still at three out of four. Looking forward to being able to control the stepper to tune the vacuum variable capacitor and calibrating the controller, and perhaps opening myself up to a new level of complications. At least the well-commented code should be a smart guide.

In the meantime – getting way ahead of myself now – I’m thinking about drilling two holes in the loop to take the stepper wiring, possibly via a length of CAT 5 cable. People seem to think the twisted pairs work well with stepper windings. Not sure what type of connector to use in VVC part and lower down alongside the feed loop SO-237.

I also need to get a couple of plastic pipe ends to fit the PVC tube cover Henrik gave me to weatherproof the stepper and the tuning capacitor.

Then I need to think about a rotator and maybe a tripod roof mounting. And a way of remotely controlling the rotator. Anthony K3NG most probably.

At least it looks like it might be ready to take away to Tooraweenah for a field test in March!!!

Magnetic loop fault finding breakthrough

Maybe my mistake has been not to refer more directly to the actual code. Over the last 24 hours I have been steadily working through the last two years of email conversations. Loftur keeps the group informed of each new version of the firmware and the newly added features. Sometimes he jumps the gun, but then releases a newer version shortly afterwards.

Well, well well! I think I have discovered why the final part of the loop controller is not working and perhaps why the A4975 stepper motor driver ICs are running so hot. I have just been doing some continuity testing working back from the stepper motor connector, K1 and noticed that there is continuity between C11 and C12 and also between C15 and C16, and there is no continuity between the legs of say C16 and C14 connected T2 etc, all of which implies that I have oriented the common mode chokes, T1 and T2 exactly 90 degrees out from where they should be – I think and I hope… They are both surface mount with four points of contact(!!!!), so there might be some interesting re-working to be done. I actually have one spare if I bugger one up. Also I may need to replace the eight 10nF ceramic bypass caps nearby. But this is progress of a kind.

Loop Controller PCB before 'Aha!' moment
Loop Controller PCB before ‘Aha!’ moment

I wonder what made me install them this way. Surely not simply the orientation of the writing on the package!?!?!?, or the oblong shaped pads on the PCB?!?!?!?! Two visual cues to be misled by!

I suppose I should be optimistic about the transformers being okay if they still show continuity… There’s a big lesson here about using the schematic during the construction and not simply populating the board. And maybe even being curious enough to actually turn the component over to see its magic revealed and be inspired to orient it correctly.

The exterior of the common mode choke with two 51uH windings
The exterior of the common mode choke with two 51uH windings
Inside the common mode choke clearly showing the two 51uH windings
Inside the common mode choke clearly showing the two 51uH windings

And now of course when I look back to page 8 of the BOM and building instructions I can clearly see the correct orientation of the transformers. Oh me oh my!

And I have just noticed during my rapid review of the emails on the Loop Controller list that on 8 May last year, while advising someone about testing the sense of the stepper winding connections, Loftur dropped this comment, “Also, have a careful look at the circuitry around the A4975 stepper controllers. One fairly common mistake is to mount a common mode choke incorrectly.” I’ll say!

Magnetic loop controller – home stretch

After a long break I have finally got back to completing the Magnetic Loop Controller designed by Loftur Jonasson that I started almost two years ago now. I am using his original PCB version from June 2014 which indicates how long I have been intending to build this device!

I noticed a small SWR board kit based on the kitsandparts kit offered via the Loop Controller list by Frederik ON5IA.

I ordered a couple, but while I was waiting I checked where I had got up to with my original build.

I completed my original SWR bridge board based on the kit from kitsandparts.com. I worked out I could use the same larger gauge bus wire (as used for the single turn in each ToroidCalc1 transformer) to securely and rigidly mount the SWR board to the inside of the BNC sockets. I discovered later that one of the joints to a BNC centre was intermittent, ie not actually soldered!

With the Controller apparently complete, I connected a 4-pin socket to the four wires coming from the stepper motor on the loop and attached the stepper to the VVC with the shaft couplers. I also spent quite a bit of time confirming the correct way to connect the main PCB to the SWR board, to the serial port, to the Stepper motor as well as all the other connectors and controls mounted on the front and back panels of the enclosure.

While the Teensy appears to be working as expected there was no apparent connection with the radio or the loop!!! So far the score is just one out of maybe four! (I suppose my four targets are 1. Teensy controller, 2. Serial connection with the radio, 3. Connection to the Stepper motor and 4. SWR Autotune.)

I did notice scratchy noise due to the dodgy non-joint on one of the BNC sockets. Also the Controller would occasionally just stop and reboot, especially if the enclosure was tilted slightly!

I checked continuity of the earth/-ve line. Seemed intermittent. Noticed I had used nylon stand-offs to mount the PCB. But there was a connection to the metal enclosure between the PCB ground plane via the heatsink tab of the 5 volt regulator. Decided to install metal stand offs. Needed added height of one washer to match original height for the USB socket to match the hole in the rear panel. Also confirmed that the -ve connected directly to the enclosure via the power socket mount.

Properly soldering the antenna lead eliminated the scratchy noise from the radio.

I had last installed firmware version 3.04. I updated the Arduino IDE and re-installed the Teensyduino libraries and then installed the latest version 3.08 of the Loop Controller firmware. No actual difference with connection to radio or loop.

Reading over the last couple of years of conversations on the Loop Controller Yahoo group I came across one post where someone had experienced problems connecting to their KX3. Loftur replied with a check about the way the TXD and RXD is connected to the serial lead. Mine was wired opposite to his description, as I may have confused the order on the serial header and the serial socket. Flipped the TXD and RXD connections and immediately I was able to see the frequency readout on the LCD update after a change on the KX3. Satisfying! A review of the schematic confirmed I had misread it earlier.

This was with the Elecraft K3 Auto setting on the Controller. The controller responds to changes of frequency on the KX3 but only after there is a change. And the update of the information in the frequency display feels a little laggy and as if the update periods are longish. Switching over to the polling method Elecraft K3 Poll the update is much faster and smoother. It also recognises band changes before a change in frequency. (Perhaps I need to change the KX3 setting for AutoInf back to its default). I am using a standard stereo patch cable between the serial port on the controller and the Acc1 socket on the KX3.

One step at a time! Of my four targets then, it looks like we may have achieved two!!! Now just SWR & Stepper to go.

I suppose I should be able to test the SWR board. I have a dummy load connected to the controller ANT BNC. Perhaps I could even calibrate the Power and SWR???? I have checked the ML.h file in the firmware package and read the section in the BOM and Building Instructions pdf (page 13) about the settings to enable the Power/SWR and Autotune options. #define PSWR_AUTOTUNE had been set to 0, so I set it to 1 and verified the program and re-programmed the Teensy. All pretty speedy. And it works, but because frequency ranges have not been set yet it seems to read most frequencies as out of range.

Definitely making some significant progress today!!!

Now to attack the issue with the lack of connection between the controller and the stepper motor.

I examined the solder joints of the two common mode chokes and tested them both for connection and continuity. I have also noticed that the two stepper driver chips are running very hot, way too hot to keep your finger on them for more than a touch. Also there’s probably a static discharge risk there too. Given their cost is only about $7 for these two ICs, it might be worthwhile replacing them, while taking proper antistatic precautions. It would be good if I could discover any other reason/s there is no linkage to the stepper and more importantly why they get so hot so quickly. I hooked up another NEMA 17 stepper motor directly to the board and there was again no movement, but the chips immediately heated up. With the stepper disconnected the chips still heat up.

Time to read more of the manual. That’s the only thing lacking here – a comprehensive manual pitched at dummies like me. I suspect a lot of the wisdom about using this device is waiting to be discovered in the email discussion group. I am slowly working my way backwards over the last almost 2 years I have been subscribed to the list and so far, I’m up to conversation #214 of 265 conversations, that is I have 214 to go! It would be great if you search the conversations directly from the group page… must see why I can’t do that…

U3S + 12V = Fried U3S

Well I have now printed a label for the power connector on the U3S enclosure saying 5V, but it’s a little too late for an assortment of components that inadvertently were subjected to not just 12V but closer to 14V! Yikes.
From consulting the FAQ on the QRP-Labs site at https://www.qrp-labs.com/faq.html I have learnt that I’m am by no means the first to commit this blunder! In fact – as the FAQ on this topic reveals – so many have trod this well-worn path that there’s a pretty good understanding of what needs to be repaired and what probably survived the onslaught of unwanted volts.
Net result as I understand it is:
  • the main processor, a 28 DIP ATmega328 chip is definitely fried! I do have the option to program a blank one or order one already programmed from QRP-Labs.
  • The LCD display has been zapped.
  • The relay on the main Ultimate 3S board has most likely been burnt out as well by the higher voltage.
  • It’s supposed to be unlikely that the Si5351A Synth Module has been damaged.
  • On the QLG1 GPS unit it’s probable that the 74ACT08 level converter chip has been destroyed by the high voltage because of the way the LED indicators are behaving when power is applied. Yellow is not pulsing once a second as it should…

This is such an unrare event that QRP Labs have created a product especially for people like me – the QCU QRP labs control unit which components most likely to need replacement after applying the wrong voltage! It includes the 16 x 2 LCD module, 20MHz crystal, BS170 transistor, buttons, resistors, capacitors, hardware and all connectors (4-way sockets, 10-way sockets, 16-way plug/socket for LCD) that are used in the Ultimate 3S kit.