Blog

Viewing posts for the category hack

Why? Oh Gods! Why?


$ mv .ssh/ .ssh.old/
$ python
>>>import gnomevfs
>>> gnomevfs.get_file_info( "ssh://aigarius:password@aigarius.com/home/aigarius" )
Traceback (most recent call last):
File "<stdin>", line 1, in ?
gnomevfs.AccessDeniedError: Access denied
>>>
$ ssh aigarius.com
The authenticity of host 'aigarius.com (85.254.216.40)' can't be established.
RSA key fingerprint is 6d:29:c0:f3:d0:84:c9:a9:d9:4c:7e:e3:1a:18:a2:e2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'aigarius.com,85.254.216.40' (RSA) to the list of known hosts.
aigarius@aigarius.com's password: *******
[...]
aigarius.com$ exit
$ python
>>>import gnomevfs
>>> gnomevfs.get_file_info( "ssh://aigarius:password@aigarius.com/home/aigarius" )
<gnomevfs .FileInfo 'aigarius'>
>>>

Bit horizon

A small idea for increasing the performance of peer-to-peer communications for highly popular files.

Middle-clicking files

A though came to my mind just now - the method of copying text around by just selecting it to copy and using middle-click to paste it. So I wonder, why could one not apply the same for file operations in Nautilus? I would love to just select and middle click to copy files and maybe a Ctrl or Alt key could be used at the paste stage to switch to moving files instead of copying.

IPW2200 on HP nx6110

If you thought that sticking a supported miniPCI wireless card into a HP nx6110 notebook and getting it to work is trivial or fast or well documented, then you are quite wrong! Let me walk people trough the misery here...

Testing blogging from GMail

Migrating from Blogger to your own Mnemosyne blog

Firefox backing up.

Firefox backing up.
A bug is bugging me up, but it is unclear to me what is the cause, so no reporting or fixing is yet in sight - whenever I scroll up in Firefox using the mouse scroller there is ~10% chance that the "back" action will be executed. It has happened to me a lot of times now, and I haven't noticed similar symptoms in other programs.
Ubuntu Dapper, FF 1.5, USB mouse, HP nx6110 notebook. Any advise from the lazyweb?

Another bug has pissed me enough to star...

Another bug has pissed me enough to start debugging. This time it is Totem-xine crashing on startup in Ubuntu dapper.

The first thing is that you cann't rebuild totem from sources multiple time after ubuntu patches - ubuntu uses dpatch to patch something in automake files and after the build has been run, the unpatch fails thus preventing a rebuild, doh! Worked around that by removing that patch. (Bug not reported yet)

After installing totem-gstreamer, my main suspect is the change to the statusbar, that look very recent. Could it be that Totem developers forgot a critical fix to the xine backend? Could it be that the treat xine backend as a ... second class citizen? To what? To that GStreamer? I tried to use GStreamer, I really did, but there are a few tiny issues: 1) it doesn't open even half the files that xine does, 2) within 5 minutes of a movie audio-video can easily get out of sync by 5 seconds. I have never seen A-V sync in xine. Ever. I love telling our Windows using frends that my movies "just work" with totem-xine, please do not take that away!

Anyway - back to the bug we go.

As we have a clean crash, I recompiled totem with debugging symbols ("DEB_BUILD_OPTS=nostrip,noopt debuild -us -uc") and run with gdb. When totem crashed, I got the code line, where it happened:

(totem:4608): GLib-GObject-WARNING **: invalid cast from ` ' to `TotemTimeLabel'

Program received signal SIGSEGV, Segmentation fault.
0x08068659 in totem_time_label_set_time (label=0x8199a60, time=0, length=0) at totem-time-label.c:69
69 if (time / 1000 == label->priv->time / 1000

Now, that is interesting, lets see, what we have here - time is an int, so no segfaults from there, but label is a TotemTimeLabel. Hmm, that error now makes sense. And when we take a look at label->priv, it appears to be a pointer to TotemTimeLabelPrivate with an address of 0xffffffff. That's the problem, now we only need to backtrace trough the program and find the bug that is causing that.

Well all looks pretty nice - there is a "tick" event in the player that calls the time update. Not really clear, why there is such a discrepance between GtkLabel and TotemTimeLabel or why this structure is not inicialized in time. More strange is that gstreamer backend never calls this function. Wierd. Let's see what happens if I just return from it without doing anything. Does not help - now statusbar is crashing.

Let's try it from another angle - it worked before. Nothing much in totem changed since release of breezy. Installing the version from breezy, it works fine. Recompiling the version from breezy on dapper - crashes. Ouch! It looks like xine backend of totem has not been ported to that new crazy Gnome 2.12 thingie, like gstreamer backend was. Strange - that is a backend, it should not be dependent on the frontend, no? Anyway, it is not something I can do - I will have to install the breezy version, hack some dependencies to make it no conflict with one optional library and then file a critical bug on totem for breaking the xine backend.

But even that will have to wait 'till tomorrow, sleep is of the essence, anywere.

Do you want to hear the most incredible ...

Do you want to hear the most incredible "it's not a bug - it's a feature" story ever?

After shooting hundreds of megs of RAWs with my Canon 350D last couple of weeks, I noticed a very strange thing - importing this large amount of files from my camera into F-Spot took ages. F-Spot ate memory in tens and hundreds of megabytes and never returned it back to the system. Well I blamed it on Mono and went searching for a better way. Then I found out that command-line C program gphoto also take the same horrific amount of memory to import my photos. I saw that to download 900 Mb of photos (~250 photos) photo memory use went up to ~910 Mb (2 Mb were shared). Luckily Linux managed to swap out part of gphoto, so I could finish the download with my 512 Mb of real RAM and a 1 Gb swap file. I googled and founds tens of bug reports on this - first of them as early as December 2004. Ouch.

Well - let's see what the problem is, shall we? Some bugreports reference a bug in gphoto's SourceForge bug tracker where a users reports that downloading a 250 Mb video file takes 250 Mb of RAM and developers reply that unfortunately that is the limitation of current infrastructure and it is very hard to fix. Bumer.

But wait! He says that downloading ONE file takes a lot of RAM. This limit should not exist when downloading multiple files - we should be able to drop information about previous file as soon as we start downloading the next one, right?

Ok, lest see, what really is going on there. Downloading source of gphoto. Looking at it. Seeing a lot of mess. After around 10 minutes I start to understand that there is a table of option names and functions and the real job is doe by command line parser who calls a function as soon as he encounters a proper parameter on the command line. :P After 3 more minutes jumping around the code I finally get to a function that gets called to download a single file. Looks pretty easy:


  • take a CameraFile pointer

  • pass it to gp_file_new() for inicialization

  • pass it to gp_get_file() to get the actual data of file (download happens here)

  • pass it to gp_write_file_to_file() to dump the data to a file on disk

  • pass it to gp_file_unref() to free the data


Looks all fine and dandy so far. However I see the memory use that suggest that this last operation does not happen as it should, so I search for the gp_file_unref() function. I do not find it in gphoto source, but as I soon figure out - it is in libgphoto2. The function is pretty straight forward - the reference count of the structure is reduced by 1 and if it has reached 0, the structure is freed from memory via gp_file_free() function.

Hmm, I wonder what will happen if I replace gp_file_unref() with gp_file_free() in gphoto? After a quick compile and installation (I thank the Gods and all DD's for the wonders of "debuild -us -uc && sudo dpkg -i ../gphoto*.deb") I ran gphoto again. Wow, it now only consumes 8-16 Mb of RAM and not 900. The files downloaded fine, but in the end glibc made a lot of fuss about "double free". What does that mean? It means that someone managed to get a reference to our MemoryFile and didn't give it back. Naughty boy!

We only call three functions using that pointer, so it should not be hard to trace them trough the source to see what they do. The gp_file_new() function looks good, it sets reference count to 1 always. gp_get_file is more complex - I get to crawl through a lot of strange redirects to all levels of gphoto architecture. At one point I get a bit alarmed as I see a local variable called ref_count, but then I see that the code just stores reference count there for safekeeping while data is copied from another object and right after that copy reference count is put back safely. After all that I get to the end of the gp_get_file function, just a couple thing left - cache the result, clean up and return the file. Wait a minute ....

CACHE?!?!?!!

$(&@($^@#$(^@&^$(#&$@#(&$(@#$&!^&$^@*!(&$#(@& !!!!!

It appears that someone thought that it is a good idea to use a gig or so of my RAM for cache, just in case if I would like to download the same photos the second time around in the same program call. IT IS NOT!

Results: one line patch, one NMU building for upload, one *very* long bug in upstream bug tracker, one developer quite upset and not too convinced about the correctness of free software ways any more :P

Little tiny side note: F-Spot rules as a...

Little tiny side note: F-Spot rules as a photo organiser: I can import my photos from the camera, tag them, print them, export them too photo albums, BUT there are two tiny issues:
* why cann't I start GIMP on a photo and automatically get a new Version of it when I save (preferable a separate version for each save operation)? or why can't I just say that photo 1 is a version of photo 2?
* why exactly does F-Spot consume over 250 Mb or real nonshared memory during photo import from camera and almost 150 Mb of memory at all other times?!?!?!!?!

grumble, grumble, grumble, must file bug reports, grumble, grumble...

Recent Posts

Archive

2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005

Categories

Authors

Feeds

RSS / Atom