QorIQ Linux® Software Development Kit

If you follow Linux news, you have probably noticed confusion about what makes a good Linux BSP.  I’m reminded of an old friend in IT whose definition of “released” meant that the program worked at least once on his personal workstation.

While the tech industry drones on relentlessly about quality, the fact is that even after Linux code reaches millions of lines, many software engineers still expect that the code is only reference code and hasn’t been tested thoroughly. More than that, most people believe that when you adopt Linux, you’re on your own – without support.  Companies often task their engineering teams to validate, harden and test the code before it’s released into production – this critical step can drain your precious time and resources.  This is why we took the approach to develop a Software Development Kit (SDK) that helps you shave months and months off your development schedule.

What is our SDK really? Is it just a BSP?
The tendency for most customers is to call the SDK a Board Support Package (BSP). It’s more than that.

While the thought has much truth in it, the fact is that our SDK is really much more than a BSP. I will use BSP where it makes sense in context, but when I say BSP in context, I assume the BSP is created from the SDK. SDK means multiple BSPs in many ways.

What’s in a name?
It really starts with the name.  When our software teams wanted to support QorIQ Multicore Processors, and PowerQUICC communications processors built on Power Architecture® technology, with Linux BSPs, they put lots of thought into how they would do this.

Since we have a long history of common IP blocks that are rendered in our silicon, we created multiple BSPs out of a single body of work. The body of work is similar to a library of Linux IP that would support the silicon IP. All an engineer would have to do would be to “assemble” the Linux IP pieces in the same configuration as the processor to create an overall processor-level BSP.  To support this overall, all-in-one library of Linux IP, we called it a software development kit. Think of it like this – you get the software development kit – you add some of your own IP, then press a button (an exaggeration) and shazzam – you get the ideal BSP for your board.  So now you know why we call the SDK “the SDK.”

We also assembled software tools and other Linux software along with the Linux IP for the processors, so you get an all-in-one starting point for Linux development for our communications processing platforms and multicore SoCs.

But that isn’t the only place you can get our SDK. In fact, if you think about where you can get the BSP (I shifted gears on you – sorry), you might learn something about us.  One thing that separates us from the pack is that we are a company that strives to develop much of our software completely in the open.

How would you know we develop code in open source?
You would know this because you can find some of the most recent Linux code for our communications processors directly on kernel.org, and now Yocto.  How recent is our code in the public? Well, on our GIT, it is very, very recent. On places like kernel.org, we are one of the top contributors.  We submit our code as a release candidate to kernel.org (and others), and we have a strong track record for getting code accepted into the mainline.  I’m answering this question this way because it is important to understand that you don’t just jam code into kernel.org. It goes through a vetting process. It isn’t automatic.  But you can test us on this… our code is pretty darn recent. Many, many of our customers simply point to kernel.org as their starting place for development.

Our networking software development kit, in-depth
Let’s dive into some details and see how we might be a little bit different than others. 

Accelerators and specialty silicon IP
One of the really neat things about our silicon IP is that our communications processors have specialty IP in them that you simply can’t get from your corner semiconductor foundry store.  This IP literally has thousands of development years. It has the cumulative feedback of thousands of customers and tens of thousands of software developers. The numbers are staggering. We’ve had feedback from all different industries. When you get a communications processor, you are getting the cumulative wisdom of potentially hundreds of thousands of engineers.  That creates a special challenge for the Linux developer though. It isn’t reasonable to expect someone who knows Linux and a little bit about communications to properly support a specialty peripheral that has been undergoing constant improvement for 10 – 15 years. The point is: When you start with our SDK; you can bet you are getting the LATEST and most modern support for some of the world’s most advanced semiconductor peripherals. Let’s go over just a couple of peripherals.

Security
The answer is probably “yes.”  The question is “Do we accelerate [insert your favorite esoteric cryptographic algorithm here]?”

How do we do this?
We maintain a cryptographic center of excellence. We don’t call it Area 51, but as I write this, it strikes me as funny because it isn’t too far from Area 51 – hint, hint.  What they’ve created is a common security IP block which we have used over and over again on many, many semiconductors. Not only do we use this center of excellence for developing our crypto acceleration engines, we use them to provide security consultation to other parts of our company. It becomes a virtuous circle. We use them more, they learn more, they learn more, share more and then we use them more.

The output is an amazing piece of programmable crypto acceleration that you’ll find across many solutions, including QorIQ communications processors and i.MX applications processors. We’ve even created a new standalone variant called the C29x family crypto coprocessors because customers sometimes want access to our engine, but for some reason or another don’t need a fully loaded processor.

I had one marketing person get frustrated because he created this huge spreadsheet full of all of the combinations of security algorithms he wanted supported, and the software team wasn’t filling them in fast enough. But what he failed to appreciate is that the crypto acceleration engine is programmable. Wanting all combinations of every algorithm under the sun instantly is like saying you want every single program you could think of on your PC … right now!

So, here is the million dollar question: If you want accelerated crypto operations on your Linux-based project and you’re using a QorIQ communications processor, how would you expect anyone but us to support the crypto in the best way?  Think about it. Our software engineers have immediate access to the engineers behind our crypto engine – and when you partner with us, you can TOO. 

Communications
There is no way I can do justice to the communications IP from us. Not in a blog article. Not even in a book. But maybe some short notes.

We have really changed the history of networking silicon when we introduced the 68HC302 processor. While for many that processor is simply something that happened in the past, the truth is that the 68HC302 lives on. That enlightened processor became the lightening rod for learning how to accelerate Internet and Ethernet packets. We have examined every bit of every line item in every algorithm and learned how to best reduce it to silicon, while retaining all of the software flexibility. We didn’t do it all at once. We started with our communications processor (CP), then created the DPAA and now, far from done, we’re once again innovating by creating an entirely new way of thinking about packet-based communications with our Layerscape architecture. Through all of this, we had the first Linux running — on our communications processors. We will always have the first experience with putting software on the IP.

Quality
Even though we may have the most experience with our silicon IP, many customers wonder “How good is your Linux SDK, really?”  In my mind, I equate quality with testing, but it really is much more, for example, it is the process you use. Let’s go into what we do behind the scenes with our SDK.

Test Environment
Every SDK is tested rigorously before we release it. By releasing, I mean that we make a tested snapshot available. We do this monumental task approximately twice a year. As monumental as our releases are, we do a considerable amount of testing, in far less time than you might expect. Why? 

Over the years, we decided that to be efficient in Linux development we needed to create the ability to manage multiple Linux kernel versions, multiple boards, multiple chips, multiple tests and keep track of all of these multiples. The problem is a geometric progression.  As a result, behind Linux development is a very large and interconnected Linux development and test environment. Within reason, at any one point in time, a software engineer can connect to a board contained in one of our worldwide board farms (more on board farms later), create a BSP from any number of different Linux kernels, program the kernel into the board’s flash memory and then stimulate the board any number of ways. Finally a fully capable CodeWarrior multicore remote debug session is really no problem. If we wanted, we can even hook into our IC design systems to help match the design to the upcoming software even better. Since a single software engineer can do what I described, it turns out that we can also do this with automated programs. Because we can do this with automated programs – we do.  In fact, we do this all the time. We do 24/7 testing. We are constantly monitoring key performance parameters, and when engineers check-in code in any one part of Linux, it is predictably exercised thoroughly with a fairly short wait. By the time our code gets to kernel.org, it has been ran — and ran and ran. We also make decisions on the kernel versions that the code should run on. This means we are testing on the backward and forward ports that we have as well. Quick note on debugging: While there are many development tools on the market, the one thing that CodeWarrior Development Suites for Networked Applications gives you is the ability to call us and get processor-Linux-debug support with no third party involved.

Custom Linux tests and more tests … and even more
It is no secret that in Linux there are lots and lots of tests floating around. It is no secret that we will use as many of these as possible. But if we just tested using the test suites you would find on the Internet, then we would be no better than the next company. So we have written a few tests of our own. Several thousand to be exact . Even better – we create our own reference designs. One, as an example, is the network attached storage (NAS) reference design and we run the NAS reference design code through a few paces as part of our testing. So, when we say we write a test, sometimes it is an entire application. Some of our tests have a manual component to them. For example, testing hot-plugging USB is a manual test. However, we keep track of our percentage of manual tests and are adding equipment all the time so that as many manual tests as possible can be completely converted to 100% automated tests. When we run these tests, we monitor key performance indicators. We track these indicators regularly to make sure that they stay within tolerance.  Each test has key performance indicators, if applicable, as well as expected results.

Production test equipment
I remember going to a customer site several years back and seeing the customer hook up a beat up laptop via Ethernet to their board under test. Then they started some program, I think maybe even it was still named “a.out”,  to reflect incoming packets from the board under test back to the board under test. This was the extent of much of their testing. Nothing could be farther from the case. We both purchase and lease production network test equipment. These are not little rinky dink lab PCs. These are industrial strength pieces of networking test equipment, capable of injecting all different types of packets – voice, data, streaming, video and specialty, including injecting packet errors and many other things that make sure our processors handle your packets the right way. We don’t have just one or two of these things lying around. When we think of network traffic, we’re thinking about train car loads of traffic. We measure our capacity in Terabits. The point is, our SDKs are tested with production level traffic.

Multiple boards
Earlier I discussed the notion of a board farm. A board farm is a room full of interconnected development boards with network managed power, reset, console, critical IO and network accessed JTAG. We keep on hand boards that are very old as well as boards that are very new. These boards are accessible for automated testing, as well as for individual developers. You might wonder how this helps the quality of the SDK? What it does, is it makes it so that nuances that might normally be missed when a piece of code only runs on a single board get exposed when running on many boards. It is just one more way we try to make the SDK higher quality than anyone would ever expect. Do we support all boards that we’ve ever created for networking? The honest answer is “no”. Since this is a geometric multiplication problem (Linux kernels supported * boards * software architectures * etc.) we eventually will allow older boards to stay on an older SDK. If a customer wants to get an older board updated to the latest SDK, we do engage in custom forward ports. But as a general rule, we’ll manage 20 of our latest boards in our latest SDK.

Multiple CPU Architectures
When you think about the fact that our processors support multiple CPU architectures (32-bit Power Architecture® e500 core, 64-bit e5500 and e6500 Power Architecture cores, 32-bit ARM® Cortex®-A7 cores and 64-bit Cortex-A8 cores), you would expect that it would be a problem. No. What it has allowed us to do is to exercise how Linux compiles between the different architectures and to learn how to manage and accommodate for the differences. For example, our tests “just know how” to map themselves from 32-bit to 64-bit tests. They also need to be created to handle the differences in timing without causing false failures. By squashing out as many differences as possible by conquering them, we can utilize this to make a better, more useful SDK that makes better BSPs. In some ways, you are more likely to be chip independent by using our BSP than a BSP that only supports a single architecture. 

Multiple versions of Linux
The way we handle Linux is that we maintain large repositories of different versions of Linux – sort of like a release history file. We’ll watch Linux closely, and about once a year we’ll sync to the latest kernel that is going on Long Term Support (LTS – a Linux term). Then we’ll refresh that particular kernel version in about six months. This keeps us very fresh with the latest kernels, but also does a great trade-off with our customers in terms of making sure our SDK is solid as a rock. But if you think about it, what it means is that we have forward and backward porting software built into our DNA. Our automated systems wouldn’t work well if we didn’t have this part figured out. And knowing how the code will have to travel between kernel versions means we can write the code knowing this – ergo code that simply just “works better”. 

Multiple Software Architectures
The last quality item I’ll discuss is the fact that we are as far along with virtualization as any semiconductor company in the world. As previously noted, our SDK has to be able to accommodate multiple software architectures. It’s important to note that we actually TEST all of the architectures and check the key performance indicators for each. In other words, whether you are running SMP, AMP, mixed SMP/AMP, Hypervisor (two different types), or complex guest OS situations, we’ve already tested most of these for you. Obviously, for guest OS testing, we use Linux due to licensing and availability for our engineering teams. Since we test so many software architectures, we also have to write the tests that can work in multiple software architectures as well as the CPU architectures.

I’m hoping that by now you realize that we are thinking about the Linux quality at a level that is hard to beat.  It would simply cost hundreds of millions of dollars to match the work that have done. 

Experience
So far I’ve discussed our technology in detail, but you get so much more than you’d expect when you choose us. One of the big “wins” you get with the SDK is experience. All of the monumental work I’ve described was performed by dedicated engineers. Most people know we have many, many chip designers working on our state of the art communications processors. But what you might not know is that the ratio of software developers to chip developers is currently 1 to 1 and growing. The point is: To leverage our high performance, feature rich, security-enabled processors, you need software and we are committed to the complete solution. Just to underscore our engineering experience, you can go to kernel.org and find the yearly report called “Who is writing Linux”.  In the 2013 report, we ranked as the #13 contributor in the world, alongside networking and IT leaders like IBM and Cisco. You can’t be a fly-by-night and be on that list. In fact, you have to be pretty good since the keepers of Linux scrutinize contributions. Related to our experience is the emphasis that we put on ease of use when it comes to the customer’s experience with our products. We really want people, even hobbyists who are working on projects for maker fairs, to be able to utilize our processors. As a result, we are committed to the Yocto project. We experienced first-hand the difficulties of supporting all the multiples (CPUs, CPU architecture, kernels, accelerators, boards, etc.) and we brought that experience to the Yocto project. We are literally trying to make it so that you simply get a recipe for the board, type a command to execute the recipe and you get instant BSP – working perfectly for your board. While I’m oversimplifying, I’m not understating the goal and wisdom we bring to the table.

Security
One area where our technology goes deep is in security. The very word sounds so specific, but it isn’t because it is such a multifaceted topic. You have a need for security. They need to manufacture, deploy, boot, execute and communicate in a way that keeps bad actors from infiltrating their system. Especially when it comes to security, our SDK has the collective wisdom of all of our customers, our developers and our semiconductors. We’ve thought through the entire value chain. Even with Arm Technology-Based Solutions, we’ve wrapped them in IP, which means that the TrustZone technology from Arm gets all the benefit of our proven manufacturing and boot logic. If you are struggling in making the entire value chain secure and you have concerns, you might consider calling us.

Now that you know, can you afford NOT to consider our SDK?
The Linux SDK for communications processors is simply NOT your ordinary Linux BSP. It really is so much more. Ask your Linux developer what they are doing on Linux. Ask them specific questions about what you’ve learned here. Then ask yourself… what if you got in a bind? When the going got tough who could you rely on to answer your questions? What if you needed help in any of the following areas? Who would be better suited to answer your questions?

• How do you forward port an older BSP to the latest version?
• How can I increase the performance of my system?
• What do I need to develop products for long-term production?
• How can I increase the quality of my system?
• How can I lower the power consumption of my system?
• Can you help me find a terribly deep and difficult bug in my system? (we have tools and techniques that I simply didn’t have space to mention here)

It will be us, the company that knows Linux inside and out, who knows their products better than anyone on the planet. Finally, because you really can lean on us when needed, because we have so much to offer, and we now invite you to ask us if we can help you with your Linux-based projects.