So I had a problem; when I left my previous employer, I had been using my iPhone to interface with work email. That worked great. But one problem that I face after I left was that all my contacts were in the Active Directory/Exchange address book system. I asked for, and received, a CSV of my contact info when I left.
Tomorrow is my first day with the new gig. I resigned from my old job about ten days, and had a nice break. But now it’s time to get back to things. I’m a little apprehensive about this whole thing, as this is a new way of working for me. Changing gears in so many ways after my long time at PopCap (4.5 years – Wow) is filling me with all kinds of feelings of volatility. What happens now?
I’m eager to get into the swing of things, and past the anxious new beginning with this. Until tomorrow.
Thanks to flickr user redjar for the image.
I hate airports. Not flying, flying I love. But the process of dealing with airlines, and luggage, and most especially the TSA/Security Theater aspect of the process drives me crazy. I flew down to SJC this week for a technical conference, MySQLConf. It was a great conference: I met neat people, I learned new things, I got excited about tech again. I loved that part. The flight down was fine. Easy, no problem through the airport. Only the usual anxiety about the process.
CC licensed image via alancleaver on flickr.
Today I went to breakfast at Voula’s Offshore Cafe in the University District. I had the steak and eggs with rye toast for breakfast. It was quite tasty. But this isn’t about my breakfast so much as it’s about the interesting example of information leakage.
I use foursquare because it’s fun, and to find where my friends are going for food, activities, and other things. I don’t link my foursquare to any other accounts, but other people do. This morning at Voula’s, I checked in, and saw one other person there. I’m curious, and so I look at the foursquare profile. Turns out there’s a facebook link on the foursquare login, so I follow it. Turns out she was sitting at the table next to mine. I said hello and made a comment about foursquare. Then I returned to my meal.
Her facebook profile said a lot of things: that we have quite a few friends in common, who she’s seeking, and her full name. Now, I didn’t say anything to her about any of this, nor attempt to engage in a conversation at this time. It would have been weird and creepy. I didn’t want to be That Guy, or a weirdo. But it was very much an interesting lesson in how we chose to interact online can affect us in the offline world.
When we all got out to the car, I had a conversation with the teenagers about information that you share online, and used this very real example from just now to illustraite that what you do online can have real results in the rest of your life. Pay attention to what you post, to what you link, to all those things. It’s very easy to draw connections between people via the social graphs.
There are many wonderful things that I get out of connecting my social graphs. I get to point people whom I want to connect with me to where they can find me, and how they can do so. But it bears consideration before you blithely do so. Everything connected means there aren’t any secrets.
Today we had to do a very large alter table on some production data. The table in question is big. 47 million rows big. We have to make some changes to drop a column and change some indexing to support our application. Fine, ALTER TABLE blah blah blah. But how do we keep an eye on the progress of the alter?
There’s nothing in the documentation for MySQL that speaks to a method, nor does any google searching pull up anything (except for the one InnoDB file per table patch.)
So what’s a clever systems administrator to do?
I looked at the output of lots of status commands from MySQL, and finally I came across something useful in the InnoDB engine status output. I noticed that there was an entry in the TRANSACTION section that speaks to the undo log entries. I noticed that after several hours of our ALTER TABLE running that the undo log entries value was up around 33 million. And then it occurred to me that since InnoDB does row-level operations, that this value might correspond to the number of rows that have been operated on throughout the process of the ALTER:
---TRANSACTION 0 2769988559, ACTIVE 53210 sec, process no 4115, OS thread id 1157654880 inserting, thread declared inside InnoDB 91
mysql tables in use 2, locked 2
111667 lock struct(s), heap size 13580272, undo log entries 33707735
MySQL thread id 948913367, query id 1662722204 localhost root copy to tmp table
So I wrote some bash. And some awk. And some perl, and I made a hacky little one-liner script to dump out the rate of change, the average rate of change, the % complete, and an estimate (based on average rate of change) as to when the query might be complete.
That script is below, and you are welcome to it. I have yet to wrap it in more clever perl to dump out an actual status bar, but I’m happy with it, and hopeful that this is a meaningful way of keeping an eye on the rate of change, and the progress of ALTER statements.
The script:
#!/bin/bash
(while(true); do \
(mysql -e 'show innodb status \G' | grep undo\ log\ entries ; sleep 1 ; \
mysql -e 'show innodb status \G' | grep undo\ log\ entries ) | \
egrep '[0-9][0-9][0-9][0-9]' |awk '{print $10;}' ; done ) | \
perl -e '$t = ROWS_IN_TABLE; while(1) { \
$n ++; $nn; $a = <>; $b = <>; $nn += ($b-$a); \
printf "Rate: %d, avg: %d, %0.3f%% complete, done in %d sec\n", \
$b-$a, $nn/$n, ($b/$t)*100, ($t-$b)/($nn/$n); }';
I hope you find this tool useful.
It took a LONG time for the query we performed to run, and it was very helpful to have this to gauge the time it’d take. The moving average is not the best for long-term accuracy; restart the script periodically for better views (and wait for ~100+ trips through the loop.)
It was within three minutes of accurate. It look a little longer than the script indicated (log undo entries was greater than the # of rows, but not by more than a small percentage (less than 0.01%.) The total query time was 22 hours, 9 minutes, and 33 seconds. So it was off by only 1 hundredth of a percent.
This is only valid for ALTER TABLE, but appears VERY useful. 
WOOHOO!
So the opening for my show is this evening. Wow, I’m excited.
So in June, I’m having my first art opening. That’s exciting! Over the last half-year I’ve been talking to the folks at Utilikilts about whether they’d be interested in seeing my works in their store for Art Walk. They expressed interest that yes, indeed, they would like that. So we talked, and figured things out, and now I’ve got an opening coming up.
This collection of works ranges over a few areas of interest for me: Burner Culture, People, and some other pieces. Please come see my work next month, on June 4th for Art Walk at the Pioneer Square Utilikilts store. There will be a live DJ and drinks to be had.
It’s very exciting, and I’m looking forward to it. I hope to see everyone there.
I attended MySQL Conf, and it’s been an neat conference. The most surprising thing was the news was the purchase of Sun by Oracle. This sounds to me like a play for a solid vertically integrated market for Oracle. I can appreciate that — they get Sun’s hardware and OS skills, they bring their own DB services to the table, and they get a lot of community with MySQL.
Lots of people have spoken about their fears that Oracle will suffocate MySQL, but I don’t think they’ll do that. I think that Oracle realizes how centrally important to the web community that MySQL is. I do think they’ll try to sell Oracle to high-volume MySQL sites, and that makes sense. And I think they’ll continue MySQL to bring DB share under the Oracle roof.
MySQL is hugely popular, it wouldn’t make any sense to throw away that good will.
Other things from the conference have been that I’ve run into people I haven’t seen in a while, and I’m learning things.
One of the most interesting talks I went to was Rockyou’s talk about how they built their federated platform for supporting their Facebook applications. Very neat to see their approach, especially since we’re buidling our own sort of Facebook applications, specifically Bejeweled Blitz. Good times.
I met the photographer called Julian Cash, a very nice guy located in San Francisco. He did a light painting portrait of me, which was fun. We talked about various things for a while, and he talked to me about his project that he’s working on. More to come later.
Today is the last day. Things are winding down, and I’m preparing to go home.
This past weekend was Norwescon, a sci-fi/fantasy convention. It was awesome. I put on a shoot for free out of my room, and I got a good turnout, and I had a lot of fun. It was a good experience, and good practice. Next year, I know some things I’ll do better.
I got a lot of good feedback, and pleased participants. Woohoo.
I think it’s important to do things like this to give back to the community — to let people come if they want to have a photo made, and share in a creative process. The best photos were those when the subject played along, and had ideas about what to do. I like it when it’s a collaborative project.
Anyway, enjoy. Click the image for the flickr.
-Gabriel
It’s been a long time since I’ve updated this thing. Lots of changes, some photos. Good times. 
Christmas has come and gone, thank goodness. The holidays aren’t for me. Winter is over, and I’m glad. But it’s been too long without updating. Without creating. Without feeling good about things. It’s time for that to stop, and for things to get back on track.
So, I’ve got some things cooking that’ll be good. I have a strong lead on a show in the summer (more to follow later), some leads on good ideas/concepts to push myself, and I’m biking to work now. 11 miles in the morning is good fun.
But where to go from here, that’s the thing. I have ideas, but to make them real…