Saturday, 18 May 2013

Fedora: "[Errno 12] Timeout" problem with Yum

Today I encountered a problem while trying to check for updates with yum. It was quite a while ago that I last used the command line tool, as the super annoying PackageKit was constantly reminding me how lazy I am (though telling PackageKit not to be an old maid is extremely simple, but this is another story).
Anyhow, check-update wasn't working quite well:
 [lillo@pc-lillo ~]$ sudo yum check-update  
 [sudo] password for lillo:  
 Loaded plugins: langpacks, presto, refresh-packagekit [Errno 12] Timeout on (28, 'connect() timed out!')  
 Trying other mirror. [Errno 12] Timeout on (28, 'connect() timed out!')  
 Trying other mirror. [Errno 12] Timeout on (28, 'connect() timed out!')  
 Trying other mirror.  
 Could not retrieve mirrorlist error was  
 12: Timeout on (28, 'connect() timed out!') [Errno 12] Timeout on (28, 'connect() timed out!')  
 Trying other mirror. [Errno 12] Timeout on (28, 'connect() timed out!')  
 Trying other mirror.  
I immediately thought there was a problem with the proxy configuration for yum: at work I use the company's proxy, so settings in /etc/yum.conf are required. I just commented out all the proxy-related stuff.
 [lillo@pc-lillo ~]$ sudo gedit /etc/yum.conf
 # PUT YOUR REPOS HERE OR IN separate files named file.repo  
 # in /etc/yum.repos.d  
 # The proxy server - proxy server:port number   
 # proxy=http://PROXY_ADDRESS:PROXY_PORT  
 # The account details for yum connections   
 # proxy_username=USERNAME   
 # proxy_password=PASSWORD 
At this point I checked again for updates, but things were still not working right. After some time spent on thinking about how mediocre I am, I thought of the environment variables.
 [lillo@pc-lillo ~]$ echo $http_proxy  
 [lillo@pc-lillo ~]$ echo $https_proxy  
Damnit, the environment variables. I forgot that months ago I had put the $http_proxy and $https_proxy variables in the /etc/environment file. In my case I just had to comment-out the proxy-related lines:
 [lillo@pc-lillo ~]$ sudo gedit /etc/environment  
If yours is not the case, or if you have no /etc/environment at all, there are other ways to configure your variables. Good. Now, dear yum, would you mind?
 [lillo@pc-lillo ~]$ sudo yum check-update    
 Loaded plugins: langpacks, presto, refresh-packagekit  
 bumblebee-nonfree                           | 2.9 kB   00:00     
 google-chrome                               | 951 B    00:00     
 google-talkplugin                           | 951 B    00:00     
 rpmfusion-free-updates                      | 3.3 kB   00:00     
 rpmfusion-nonfree-updates                   | 3.3 kB   00:00     
 sublime2                                    | 1.3 kB   00:00     
 updates/17/x86_64/metalink                  | 29 kB    00:00     
 updates                                     | 4.6 kB   00:00     
 updates/primary_db                          | 8.2 MB   00:15     
 empathy.x86_64                           updates  
 kernel.x86_64                               3.8.12-100.fc17      updates  
 kernel-devel.x86_64                         3.8.12-100.fc17      updates  
 kernel-headers.x86_64                       3.8.12-100.fc17      updates  
 nspr.x86_64                                 4.9.6-1.fc17         updates  

Friday, 10 May 2013

High CPU usage due to kworker

I recently solved a problem with a kworker process taking up more than 60% of one of my eight CPUs. What is kworker? Here is a simple yet good explanation. I forgot to take snapshots, but the scenario was more or less as shown in figure (source).
Before starting, please note that this is not a OS-specific issue. Read this stuff again if you're not fully convinced about it; the above links could suggest this is only a Ubuntu issue, but it's not (I'm using Fedora for example).
 $ uname -a  
 Linux pc-lillo 3.8.11-100.fc17.x86_64 #1 SMP Wed May 1 19:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux  
I don't really like to say I "solved" the problem as I see the following as a workaround, but it actually works good. I just followed what I could find here and here. So, here is what you should do if you see that something is taking up too much CPU.
Make sure the offender is indeed a kworker process. Use top, or ps aux | grep kworker if you prefer.
Find out the gpe (general purpose event) that is causing all of this:
 $ grep . -r /sys/firmware/acpi/interrupts  
Normally you should get a list of this kind:
 /sys/firmware/acpi/interrupts/sci:   162  
 /sys/firmware/acpi/interrupts/error:    0  
 /sys/firmware/acpi/interrupts/gpe00:    0  invalid  
 /sys/firmware/acpi/interrupts/gpe01:    0  invalid  
 /sys/firmware/acpi/interrupts/gpe02:    0  disabled  
 /sys/firmware/acpi/interrupts/gpe03:    0  enabled  
 /sys/firmware/acpi/interrupts/gpe04:    0  disabled  
 /sys/firmware/acpi/interrupts/gpe_all:   162  
 /sys/firmware/acpi/interrupts/ff_gbl_lock:    0  enabled  
 /sys/firmware/acpi/interrupts/ff_pwr_btn:    0  enabled  
 /sys/firmware/acpi/interrupts/ff_slp_btn:    0  invalid  
You should see a gpe with a very high value. In my case the value of gpe1B was 162, which is ok; gpe06 had instead a value of (approximately) 170.000. This is definitely NOT ok: that's the proof the problem is with ACPI interrupts.
Let's say the offender gpe is called gpe[XX] (replace the "[XX]" accordingly). Now:
 $ su  
 # cp /sys/firmware/acpi/interrupts/gpe[XX] /pathtobackup  
Note that some commands will be executed even without administrator rights, but the trick will not work (or at least not completely). So it's good to keep the admin rights until the end.
Now we should schedule a task through crontab. Such task should disable the offender gpe, and it must be performed every startup or reboot. To open crontab with vi:
 # crontab -e  
If you want to use another editor (e.g., gedit), just specify it:
 # env EDITOR=gedit crontab -e  
Add the following line to the crontab file:
 @reboot echo "disable" > /sys/firmware/acpi/interrupts/gpe[XX]  
Save, close.
To make it work even after suspend (optional):
 # touch /etc/pm/sleep.d/30_disable_gpe[XX]  
 # chmod +x /etc/pm/sleep.d/30_disable_gpe[XX]  
 # gedit /etc/pm/sleep.d/30_disable_gpe[XX]  
Copy this script in 30_disable_gpe[XX]:
 case "$1" in  
     echo disable > /sys/firmware/acpi/interrupts/gpe[XX] 2>/dev/null  
 exit $?  
Save, close, and we're done. Sweet.

Note: no luck with the cron job? Have a look at Myk's comments below...