This is the blog of John Dulaney, a hacker of Fedora, SCAdian, player of Music, blacksmith, sailor, and consumer of Bacon.
Tag Archives: Fedora
I have a bunch of Sanborn maps in JPEG 2000 (.jp2) format that are quite large. The problem is that the the proprietary CAD software I use does not support this particular file format. Therefore, I need to convert to something it does support; I chose png. To convert all the files in a directory at once, I cd into that directory and run this:
jdulaney@gefjon:~/$ gimp -n -i -b – <<EOF
(let* ( (file’s (cadr (file-glob “*.jp2” 1))) (filename “”) (image 0) (layer 0) )
(while (pair? file’s)
(set! image (car (gimp-file-load RUN-NONINTERACTIVE (car file’s) (car file’s))))
(set! layer (car (gimp-image-merge-visible-layers image CLIP-TO-IMAGE)))
(set! filename (string-append (substring (car file’s) 0 (- (string-length (car file’s)) 4)) “.png”))
(gimp-file-save RUN-NONINTERACTIVE image layer filename filename)
(set! file’s (cdr file’s))
For those of you that use Fedora to host virtual machines, especially if you are using kvm and libvirt, here are some easy tunings to put into place that will improve virtual machine performance.
vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000
These settings set the maximum percentages of dirty pages in active and total memory, increase the nimbleness of cleaning them out, set your swappiness to minimum, and reduce the chances of cpu migration. These settings go into /etc/sysctl.conf
For most, these tunings alone will really improve performance. However, if you have a system with plenty of memory and generally have multiple vms running all the time, you may want to consider putting hugepages into place. Hugepages improve the efficiency of memory utilization by breaking memory up into larger chunks, or pages, than is the default. On most x86_64 systems, hugepages are 2048kb in size.
Before you decide to enable hugepages, you will need to make sure that you have plenty of ram. A vm that is set to use 1024MB requires 576 hugepages, or 1152MB. Luckily, adding additional ram to vms or running multiple vms causes this utilization to be fairly linear. You’ll also want to be sure to keep enough free ram available for anything else you’ll be doing on your system; for most use cases, hugepages shouldn’t take up more than half of the system’s ram.
A good general rule of thumb for most x86_64 systems is to figure out the total ram, in kb, for all the vms you’ll want to use hugepages, then divide by the hugepage size (generally 2048 kb) then add thirty or so, rounding up to the nearest hundred (if the nearest hundred requires rounding down, round to the fifty). This is your starting hugepage number; experimentation may be required to adjust this for your use case.
To set the number of hugepages, once again, edit /etc/sysctl.conf and add the following line:
vm.nr_hugepages = <number calculated above>
Then, you’ll want to reboot your system to make sure that your memory gets cleared and hugepages get reserved.
Finally, you’ll want to edit each of the vms’ xmls with virsh edit <vm name> and include the following:
Then, when you start your virtual machine(s), check your hugepage utiliziation by running grep Huge /proc/meminfo
[jdulaney@volsung ~]$ grep Huge /proc/meminfo
AnonHugePages: 120832 kB
Hugepagesize: 2048 kB
“In retrospect, something like the Snowden leaks had to happen at some point, but it would have been hard to foresee the actuality in 2013. Those leaks have had a significant effect on our community, making it clear that we are under attack, but also that we arguably have the best defense available against institutionalized surveillance. We have always known that closed-source software can act contrary to its users’ interests; now the wider world is learning that lesson as well. Free software, in the end, may have an important role to play in maintaining the freedom of our societies.”
— Jonathan Corbet
Although I use Free Software because it is often technologically superior, there are other reasons, as well. With Free Software, you have a much higher assurance that you know what is actually going on with your computer.
It is well known that Apple, especially, has back doors into their products; how else are they able to access your system remotely to ‘fix problems’?
Remember Sony installing rootkits on people’s computers to monitor their music listening?
Do you really trust these companies with your secrets?
Then, on top of this, even if these companies are totally benign and altruistic, the fact remains that you have a *known back door into your system*. Who’s to say that someone couldn’t exploit this for nefarious ends?
Greg DeKoenigsberg picked me up on his way from Durham, as Fayetteville sits on the route. On arrival at the hotel, I checked in and saw some Fedora shirts in the lobby. After having a quick round with the mead I’d brought, I piled my junk in my room, and we all hit the Sticky Fingers up the street.
The Europeans decided to crash; seeing as they were pretty jetlagged, I couldn’t blame them. I, however, went to the other hotel, and saw Kevin, Maria, Xavier, and Toshio. We spent the evening catching up, eventually being joined by Jon Disnard. The discussion turned to Fedora (of course) and tended to resolve around the dual topics of the Fedora Board and Fedora Women. I also learned that my room mate, Nitesh, was among those suffering from Cancelled Flight Syndrome.
The first day started with breakfast in the hotel. I was disappointed by the lack of bacon, grits, and sweet tea. The hotels were less than a mile from the school where the actual conference was held, so I walked every day.
The conference itself was kicked off by Robyn’s State of Fedora address. What she said about working on Fedora out of love was true, at least for me. I love Fedora, and pull all sorts of hours working on Fedora. The first talk I attended was Kyle McMartin’s talk on porting Linux and the various Gotchas that can result. One amusing example from one of the other attendees was a bug that caused all email read by qmail to dissapear without a trace into a black hole, but with qmail repoting in its logs that everything was perfect and dandy. Despite the obviouse security of this arrangement (no one can read emails that are in a black hole), it was still not exactly a wanted feature.
After, I attended Jon Masters’ two ARM talks, then went to lunch with a whole crowd of Fedora folks. After lunch I went to Toshio’s Python3 hackfest, where I learned a few useful tips on testing code for Python3 readiness.
I finished up the day’s sessions with Cristoph’s Alternative Desktops talk. He informed me that Fluxbox does not count as a full alternative desktop.
That evening was the first party, during which I talked with Tim Flink some about my replacement for depcheck. We’d had some confusion on what exactly was going on, but it is now straight.
The morning keynote wound up being delayed by technical difficulties (a Debian laptop not doing the external monitor thing). As a result, I had fifteen minutes to give my first talk, which I had planned to do in fourty five. I did get to hit the major differences between virt on x86 and virt on ARM. My second talk went much more smoothly, although I did find an error reporting bug. Pretty much the rest of the day was spent doing QA-related stuff, starting out with Tim’s overview of where he wanted Taskbot to go. This was followed by lunch with some of the QA crew with a few added folks, such as our FPL (and a former FPL). After lunch, we discussed where Taskbot for a couple of hours, throwing out ideas for implimentation.
After the Taskbot hackfest, I attended the ARM hackfest, which, unfortunatelly, wound up being largely a discussion on which Red Hat folks would be getting which boards between three or four Red Hatters, leaving much of the room out. This rather annoyed me, as I wound up wasting some valuable time (I could have gone with Joseph and Tim to work more on Taskbot).
After the keynote on open source fonts, I met up with Tim and Joseph again to talk about QA stuff. The first actual talk of the day I attended was the Ansible talk. After lunch was more working on QA stuff, followed with poking around to see what sort of support for the Beaglebone Black exists in the upstream kernel. The shindig that night was at the aquarium, and I got to see Fort Sumter (where the Late Unpleasantness began) with my own eyes, though at a distance. I also took some time to check out a replica of the Hunley, the first submarine to ever sink a ship in combat.
The biggest happening on the last day was the first-ever QA meeting in meat-space. The first half was a discussion on ARM going primary from QA’s perspective. The second was Visible Cloud, which means cloud images will require QA scrutiny. Unfortuneatelly, I was not able to attend the second half due to my ride leaving.
All in all, it was a decent conference. My biggest wish was that there had been more time dedicated to hackfests, since that’s Where Things Get Done. I was not able to meet up with Kyle McMartin as I had hoped, so some llvm and kernel stuff didn’t get done.
Looking forwards to Flock next year!
This year’s North American Fedora Users and Developer’s Conference (FUDCon) was in Lawrence, Kansas. I took the train from North Carolina, with layovers in Washington and Chicago. The trip went very well: I had lunch in DC with Brian Kemp and then in Chicago I played tourist. All along the way I took photographs of trains (those of you that know me will know why) that are unusual back home.
I arrived in Lawrence around midnight Friday morning; Jon Disnard (masta) picked me up from the train station.
FUDCon itself kicked off with an abbreviated (that always seems to happen) State of Fedora speech by Robyn Bergeron, our esteemed Fedora Project Leader. In the speech, Robyn pointed out that Fedora remains strong, Fedora 18 is finally out, and that the future looks bright and shiny. Fedora 19 shall be known as Schrodinger’s Cat; there is currently no set release date due to the massive slippage of Fedora 18.
After Robyn’s talk, Barcamps were proposed and voted on. I started out with the two ARM barcamps; the first was a general overview of the state of ARM in Fedora, the second was a demonstration of the new 24 core Calxedra ARM based servers that will be used as build machines for Fedora ARM.
After lunch were several lighning talks. The most interesting for me was the one on pkgwat, which can be used to querie Bodhi on status of various packages; whether they are in testing, stable, how much karma they have, etc. I was also fairly interested to find out that the GPIO on my Raspberry Pi can be controlled from the command line. This is something that can be used to teach basic programming.
I went to Spot’s talk on overhauling the Fedora release process model, which turned out to not be quite what I expected. Spot’s proposal is to go to a release cycle similar to the old Red Hat Linux cycle: have a major release (x.0) that may not have all the polish and all the major features quite done, but gets the code out there, with the following three releases adding more and more polish and stability. I commented that I worried that this could either put extra load on the QA team and package maintainers or potentially keep out ultra new features in later minor releases, going against the fourth foundation of “first”.
Afterwards, I attended Tim Flink’s blocker bug process talk so that I could provide input and back him and Adam Williamson up on questions. The talk quickly became a Q&A/Discussion session, but considering the mix of non-QA people present, this led to some comments on the inadequacy of the current system in a few areas (to be farther discussed in a bit). I finished up the barcamp session with the talk on Fedora Formulas; a potential Ansible replacement for spins. The idea is to use Ansible playbooks to customize packages installed and config post installation of Fedora.
Friday evening I went to dinner with the ARM team, and after much revelry returned to the hotel to catch up with Tim for a while prior to turning in.
Saturday was largely spent with the ARM folks. The first half of the day was discussing various aspects of the ARM project, including push to Primary Arch, dropping support for ARMv5, adding support for v7, and the v8 bootstrap project. There was also discussion on employing new Calxedra hardware and migrating the builders from Seneca to a Red Hat facility in Pheonix. During this I explained the blocker bug process and set up blocker trackers for F18 Final for ARM and F19. More on this in a bit. That afternoon the focus was on hacking up various fixes for problem packages and some experientation. Jon Masters also gave a short presentation on the kernel on ARM devices and where things were located, which will be very useful. The Calxedra guy also allowed access to a few cores on his server, so I grabbed one to play with, setting up an rpm build environment and trying to see if I could build a couple of problem packages. I was also hacking on my Raspberry Pi, running the configure script in preperation for attempting to build GCC 3.8. I was also helping Kiara with setting up her Raspberry Pi. The networking to pull all this off was rather Rube Golberg in nature: dhcp and access to the school’s wireless was done via the Calxedra guy’s laptop, which was bridging the wireless to its Ehternet port, which was tied into an Ethernet switch. This switch had a wireless access point attached (which I was tied into with my laptop) as well as Jon Master’s laptop, the Calxedra server, and Kiara’s Raspberry Pi. I had my own switch hooked to the Ethernet port on my laptop, which was tied to my Raspberry Pi and Kiara’s laptop; her Raspberry Pi was getting power from my laptop over USB. None of this counts the other folks in the room tied into the wireless AP to gain access to the server.
Saturday evening was the always-awesome FUDPub, which featured a buffet and open bar. I moved between tables some talking to folks. After FUDPub many of us went bowling; this year I knew better than to bowl against Adam Williamson. I managed to hit the high double digits (yay!). After this we returned to the hotel where there was more socializing, some Youtube watching, and virtual machine building.
Sunday was QA day. Adam, Tim, and I discussed the current blocker bug naming scheme and how confusing it is. After some discussion, we came up with a new scheme that Adam has posted to the Test list for comment. We then started discussing how to manage blocker bugs outside of Bugzilla, but I temporarily left the discussion to explain to Paul Whalen in more detail the release process from QA’s perspective so that he can then better bring the rest of the ARM folks up to speed. I reccomended that ARM follow this process as closely as possible (with some modifications needed for ARM to remove non-applicable criteria and the like). He then had to leave for his flight. I rejoined the discussion on the new blocker bug proposal web app until Tim pinged Emily Dirsh to help him with UI design. Adam and I then finished up some loose threads with the earlier blocker tracking name scheme discussion. Adam then went off to join Tim and I met up with lmr to create a virtual machine with an autotest server to familiarize myself with autotest as it stands now. After getting that set up, I rejoined Tim and Adam for the end of the blocker app discussion before we retired to the hotel.
This was an excellent FUDCon; I did get quite a lot of work done and learned a few things that will be useful going into the future. It was also good to catch up with people again, many of whom I had not seen in a year. I am very glad to have been able to go and hope to make it next year.