Crap Nullpointer Exceptions

This is the blog of John Dulaney, a hacker of Fedora, SCAdian, player of Music, blacksmith, sailor, and consumer of Bacon.

Category Archives: Fedora

Better way of importing css stylesheets in Flask

Flask is pretty cool.  But, I ran into a little snag today where if I used dynamic urls, my stylesheets broke.  The solution is url_for in the head section of your base template (or otherwise).  You can get it to work by using the following for inserting your css:

<link rel=”stylesheet” type=”text/css” href=”{{ url_for(‘static’, filename=’style.css’) }}”>

Advertisements

Batch convert .jp2 files to png from the command line with GIMP

I have a bunch of Sanborn maps in JPEG 2000 (.jp2) format that are quite large.  The problem is that the the proprietary CAD software I use does not support this particular file format.  Therefore, I need to convert to something it does support; I chose png.  To convert all the files in a directory at once, I cd into that directory and run this:

jdulaney@gefjon:~/$ gimp -n -i -b – <<EOF
(let* ( (file’s (cadr (file-glob “*.jp2” 1))) (filename “”) (image 0) (layer 0) )
(while (pair? file’s)
(set! image (car (gimp-file-load RUN-NONINTERACTIVE (car file’s) (car file’s))))
(set! layer (car (gimp-image-merge-visible-layers image CLIP-TO-IMAGE)))
(set! filename (string-append (substring (car file’s) 0 (- (string-length (car file’s)) 4)) “.png”))
(gimp-file-save RUN-NONINTERACTIVE image layer filename filename)
(gimp-image-delete image)
(set! file’s (cdr file’s))
)
(gimp-quit 0)
)
EOF

Tuning Fedora as a hypervisor

For those of you that use Fedora to host virtual machines, especially if you are using kvm and libvirt, here are some easy tunings to put into place that will improve virtual machine performance.

vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000

These settings set the maximum percentages of dirty pages in active and total memory, increase the nimbleness of cleaning them out, set your swappiness to minimum, and reduce the chances of cpu migration.  These settings go into /etc/sysctl.conf

For most, these tunings alone will really improve performance.  However, if you have a system with plenty of memory and generally have multiple vms running all the time, you may want to consider putting hugepages into place.  Hugepages improve the efficiency of memory utilization by breaking memory up into larger chunks, or pages, than is the default.  On most x86_64 systems, hugepages are 2048kb in size.

Before you decide to enable hugepages, you will need to make sure that you have plenty of ram.  A vm that is set to use 1024MB requires 576 hugepages, or 1152MB.  Luckily, adding additional ram to vms or running multiple vms causes this utilization to be fairly linear.  You’ll also want to be sure to keep enough free ram available for anything else you’ll be doing on your system; for most use cases, hugepages shouldn’t take up more than half of the system’s ram.

A good general rule of thumb for most x86_64 systems is to figure out the total ram, in kb, for all the vms you’ll want to use hugepages, then divide by the hugepage size (generally 2048 kb) then add thirty or so, rounding up to the nearest hundred (if the nearest hundred requires rounding down, round to the fifty).  This is your starting hugepage number; experimentation may be required to adjust this for your use case.

To set the number of hugepages, once again, edit /etc/sysctl.conf and add the following line:

vm.nr_hugepages = <number calculated above>

Then, you’ll want to reboot your system to make sure that your memory gets cleared and hugepages get reserved.

Finally, you’ll want to edit each of the vms’ xmls with virsh edit <vm name> and include the following:

<memoryBacking>
<hugepages/>
</memoryBacking>

Then, when you start your virtual machine(s), check your hugepage utiliziation by running grep Huge /proc/meminfo

[jdulaney@volsung ~]$ grep Huge /proc/meminfo
AnonHugePages:    120832 kB
HugePages_Total:    3000
HugePages_Free:     2424
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Virtualization on ARM with Fedora

I’ve uploaded an image for the ARM Chromebook that gives a running hardware-accelerate virtual machine (1) as well as the checksum (2).  All passwords are fedora

To run the virtual machine, run /home/virt/virt/Fedora-18-vexpress-xfce-armhfp/boot/boot-kvm as root.  It’s a Fedora 18 image.

Notes:  The kernel does not support usb 3 or wifi; it’s a just enough to get virt working kernel, not all the bells and whistles are built in.

Qemu is built from F20 srpms.  The kernel is patched to fix lpae on exynos5.  It uses a u-boot with hyp mode from Virtual Open Systems.

(1) http://jdulaney.fedorapeople.org/chromebook-kvm.img.xz

(2) http://jdulaney.fedorapeople.org/chromebook-kvm.img.xz.sha512sum

Announcing my new project: Bringing the MIT Whirlwind I to simh

Almost five years ago, my mentor and instructor in how to program passed away from cancer after a long and full life. His software writing started with the MIT Whirlwind I almost sixty years ago. As a tribute to his memory, I have decided to write an emulator for this computer and try to get it included in the simh suite. Right now, I am researching the computer and beginning to formulate my approach to the project.

Wish me luck!

FUDCon NA 2013

This year’s North American Fedora Users and Developer’s Conference (FUDCon) was in Lawrence, Kansas.  I took the train from North Carolina, with layovers in Washington and Chicago.  The trip went very well: I had lunch in DC with Brian Kemp and then in Chicago I played tourist.  All along the way I took photographs of trains (those of you that know me will know why) that are unusual back home.
I arrived in Lawrence around midnight Friday morning; Jon Disnard (masta) picked me up from the train station.
FUDCon itself kicked off with an abbreviated (that always seems to happen) State of Fedora speech by Robyn Bergeron, our esteemed Fedora Project Leader.  In the speech, Robyn pointed out that Fedora remains strong, Fedora 18 is finally out, and that the future looks bright and shiny.  Fedora 19 shall be known as Schrodinger’s Cat; there is currently no set release date due to the massive slippage of Fedora 18.
After Robyn’s talk, Barcamps were proposed and voted on.  I started out with the two ARM barcamps; the first was a general overview of the state of ARM in Fedora, the second was a demonstration of the new 24 core Calxedra ARM based servers that will be used as build machines for Fedora ARM.
After lunch were several lighning talks.  The most interesting for me was the one on pkgwat, which can be used to querie Bodhi on status of various packages; whether they are in testing, stable, how much karma they have, etc.  I was also fairly interested to find out that the GPIO on my Raspberry Pi can be controlled from the command line.  This is something that can be used to teach basic programming.
I went to Spot’s talk on overhauling the Fedora release process model, which turned out to not be quite what I expected.  Spot’s proposal is to go to a release cycle similar to the old Red Hat Linux cycle:  have a major release (x.0) that may not have all the polish and all the major features quite done, but gets the code out there, with the following three releases adding more and more polish and stability.  I commented that I worried that this could either put extra load on the QA team and package maintainers or potentially keep out ultra new features in later minor releases, going against the fourth foundation of “first”.
Afterwards, I attended Tim Flink’s blocker bug process talk so that I could provide input and back him and Adam Williamson up on questions.  The talk quickly became a Q&A/Discussion session, but considering the mix of non-QA people present, this led to some comments on the inadequacy of the current system in a few areas (to be farther discussed in a bit).  I finished up the barcamp session with the talk on Fedora Formulas; a potential Ansible replacement for spins.  The idea is to use Ansible playbooks to customize packages installed and config post installation of Fedora.
Friday evening I went to dinner with the ARM team, and after much revelry returned to the hotel to catch up with Tim for a while prior to turning in.
Saturday was largely spent with the ARM folks.  The first half of the day was discussing various aspects of the ARM project, including push to Primary Arch, dropping support for ARMv5, adding support for v7, and the v8 bootstrap project.  There was also discussion on employing new Calxedra hardware and migrating the builders from Seneca to a Red Hat facility in Pheonix.  During this I explained the blocker bug process and set up blocker trackers for F18 Final for ARM and F19.  More on this in a bit.  That afternoon the focus was on hacking up various fixes for problem packages and some experientation.  Jon Masters also gave a short presentation on the kernel on ARM devices and where things were located, which will be very useful.  The Calxedra guy also allowed access to a few cores on his server, so I grabbed one to play with, setting up an rpm build environment and trying to see if I could build a couple of problem packages.  I was also hacking on my Raspberry Pi, running the configure script in preperation for attempting to build GCC 3.8.  I was also helping Kiara with setting up her Raspberry Pi.  The networking to pull all this off was rather Rube Golberg in nature:  dhcp and access to the school’s wireless was done via the Calxedra guy’s laptop, which was bridging the wireless to its Ehternet port, which was tied into an Ethernet switch.  This switch had a wireless access point attached (which I was tied into with my laptop) as well as Jon Master’s laptop, the Calxedra server, and Kiara’s Raspberry Pi.  I had my own switch hooked to the Ethernet port on my laptop, which was tied to my Raspberry Pi and Kiara’s laptop; her Raspberry Pi was getting power from my laptop over USB.  None of this counts the other folks in the room tied into the wireless AP to gain access to the server.
Saturday evening was the always-awesome FUDPub, which featured a buffet and open bar.  I moved between tables some talking to folks.  After FUDPub many of us went bowling; this year I knew better than to bowl against Adam Williamson.  I managed to hit the high double digits (yay!).  After this we returned to the hotel where there was more socializing, some Youtube watching, and virtual machine building.
Sunday was QA day.  Adam, Tim, and I discussed the current blocker bug naming scheme and how confusing it is.  After some discussion, we came up with a new scheme that Adam has posted to the Test list for comment.  We then started discussing how to manage blocker bugs outside of Bugzilla, but I temporarily left the discussion to explain to Paul Whalen in more detail the release process from QA’s perspective so that he can then better bring the rest of the ARM folks up to speed.  I reccomended that ARM follow this process as closely as possible (with some modifications needed for ARM to remove non-applicable criteria and the like).  He then had to leave for his flight.  I rejoined the discussion on the new blocker bug proposal web app until Tim pinged Emily Dirsh to help him with UI design.  Adam and I then finished up some loose threads with the earlier blocker tracking name scheme discussion.  Adam then went off to join Tim and I met up with lmr to create a virtual machine with an autotest server to familiarize myself with autotest as it stands now.  After getting that set up, I rejoined Tim and Adam for the end of the blocker app discussion before we retired to the hotel.
This was an excellent FUDCon; I did get quite a lot of work done and learned a few things that will be useful going into the future.  It was also good to catch up with people again, many of whom I had not seen in a year.  I am very glad to have been able to go and hope to make it next year.

FUDCon wrapup

So, I like wraps.  Beefy miracles are also good (and I got some Beefy Miracle on my whit FUDCon shirt Saturday).  Solution?  Put Beefy Miracles in wraps.

Anyway, Saturday was Barcamp Day.  After the group photo, barcamp proposals/voting was Jared’s speech.  His message was essentially that Fedora is strong and that we need to keep going.  My first barcamp of the day was on Boxgrinder.  Boxgrinder makes automated VM creation easy, something useful for upcoming AutoQA tasks.  During lunch, I had the aforementioned Beefy Mishap.  Afterwards, I went to the Fedora ARM barcamp.  I’ll be honest, it was not entirely motivated by my desire to learn more about Fedora on ARM; they were giving away hardware.  Those of us that wanted some lined up and gave pitches as to why we should get it.  Mine was simple:  I said, “I’m QA.”  Besides Adam Williamson firing me a couple of times, things went well.  I got a few very basic ideas as to where the ARM project is going and what sort of things us QA folks will need to be doing.

After that, I went to see what the Sys Admin Study Group was up to, and sat in there for an hour and hacked.  After that was the State of the Fedora Kernel address.  As I had fairly gathered, there really isn’t anything earth shattering that has taken place in the kernel recently besides the version change.  I wish the clustering barcamp had been the hour before, since I’d have liked to have attended both.  Alas, I’ll ping the barcamp owner.  The last session was Sandro’s QA session.  I went to help him out with that, and things went well, with lots of questions asked and the process explained rather well.

Fudpub was awesome.  One dude I bowled against bowled a higher score than everyone else on the lane combined, but it was still fun.  The food was good, even if the beer selection was a bit lacking.

After Fudpub, I went through a couple of Poker games.  The first I won, the second I lost, twice.  After the second loss, it was time to go horizontal.

Sunday was spent with the Anaconda team’s hackfest.  We hashed out some ideas for testing the new UI and I put in my two cents here and there.  If there is one tiny thing about the new UI you don’t like, that’s probably a result of my input, so blame me.

Sunday afternoon I rode with Will Cohen and Greg DeKoenigsberg.  We took a back road for the first forty miles.  Winding through the mountains of western Virginia with snow on the ground was an awesome sight.  We arrived in Raleigh early enough for me to catch the 7:00 Greyhound back to Fayetteville so that I wouldn’t have to sit around until 11 PM.

Overall, I’m glad I went.  I got a fair bit of work done and learned a lot that is pertinent to my future goals within the Project.  It was nice meeting up with folks I already knew and meeting others face-to-face for the first time.

First Day of FUDCon

First day of FUDCon is pretty close to over.   Today started out rather slow; I didn’t get down to breakfast until around 8:30.  I sat with Christoph Wickert, Adam Williamson, Peter Robinson, Tim Flink, and someone else who’s escaping me at the moment.

I kicked things off with the Infrastructure hackfest.  After two hours of that, it was time for lunch, although I didn’t eat anything.  This afternoon, I went to Paul Frield’s Drupal workshop, followed by figuring out where to start working on AutoQA self testing.  After that, I failed a test.  Since then I’ve been sitting in front of the fire place drinking Tatica’s wonderful rum, talking, and hacking AutoQA.

I’m looking forward to tomorrow, for sure.

FUDCon Blacksburg 2012 is nearly here!

Attention, people of Earth.  This is Prostetnic Vogon Jeltz …

Oh, wait.

Anyhow, in a couple of days I’ll start my great Fedora quest.  Hopefully I’ll roll a natural twenty and won’t have any issues in travel.  I will be riding with the amazing Andy Grimm and the great Will Cohen.

I plan to attend Paul Frield’s Drupal talk Friday as well as hosting an AutoQA hackfest.  The rest of Friday I plan to attend various hackfests and maybe do something to cloud things up.

Saturday will probably be very cloudy, as I can use all the knowledge I can get in that area.  It looks like there will be a bit of a QA barcamp at some point, and I may show up to that to mommick Sandro some.  Sunday will be largely taken by more QA stuff, especially the hackfest where we’ll be mucking about figuring out the best way to test the new Anaconda.

Well, that’s it.  I’ll let every go back to their visit from the Vogons.

John.

Fedora 17 Test Days

Greetings, y’all!

This release cycle I am the Test Day Coordinator.  That means it is my job to help you, my fellow Fedorians, to set up test days for your packages/projects.  We have about two and a half months until Alpha release (1).  The sooner I receive test day proposals, the easier my life will be, and we all know that making my life easier is a Good Thing.  The test day schedule can be found here.

Proposing a test day is very easy.  There is a guide for proposing test days, as well as one for helping with creating the associated Wiki page and actually running the test day.

If you need any help, feel free to email me at jdulaney (AT) fedoraproject.org.

Thanks much
John Dulaney

(1) Fedora 17 Schedule