fredag 10 december 2010

GDB reverse debugging

GDB reverse debugging


Let's start with a simple C++ program in the file gdb-test.cpp:


#include <iostream>
 
const unsigned int SIZE = 10;
 
int main() {
 
  int b[SIZE];
 
  // intitialize
  for(int i=0; i <= SIZE; i++) {
    b[i]=0;
  }
 
  std::cout << "done!" << std::endl;
  return 0;
}

Let's comple and run it.


$ g++ -g gdb-test.cpp -o gdb-test
$ ./gdb-test

??? It doesn't stop!


Let's load it into gdb.


$ gdb ./gdb-test
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
[...]
Reading symbols from /home/larsr/gdb-test...done.
(gdb)

Now run it:


(gdb) r
Starting program: /home/larsr/gdb-test

Still dosn't stop! Stop it with ctrl-c:


^C
Program received signal SIGINT, Interrupt.
0x08048704 in main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {
(gdb)

Let's look at the variable i.


(gdb) print i
$1 = 4
(gdb)

Looks normal enough! But strange that it didn't get further than 4... Well, let's run some more, break, and look at i.


(gdb) continue
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x080486fc in main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {
(gdb) print i
$2 = 7
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x080486f8 in main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {
(gdb) print i
$4 = 6
(gdb)

What? First i was 7 and then it was 6! How did that happen? Lets start to record, and run some more!


(gdb) record
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {
(gdb) print i
$6 = 2
(gdb)

Something has modified i to become smaller! When did this happen? Let's watch i and go backwards.


(gdb) reverse-continue 
Continuing.
Hardware watchpoint 1: i
 
Old value = 2
New value = 1
main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {

This makes sense. We are walking backswards, and in that direction we had a value of 2 which turned into 1. Let's keep going backwards.


(gdb) reverse-continue 
Continuing.
Hardware watchpoint 1: i
 
Old value = 1
New value = 0
main () at gdb-test.cpp:11
11    for(int i=0; i <= SIZE; i++) {
(gdb) reverse-continue 
Continuing.
Hardware watchpoint 1: i
 
Old value = 0
New value = 10
0x080486eb in main () at gdb-test.cpp:12
12      b[i]=0;
(gdb)

Here we see that as we went backwards, we went from 0 to 10. That means that when we ran forward, i went from 10 to 0, and it happened when we executed


b[i]=0;

Could it be that writing to b[10] overwrites the memory of i? Where are they located in memory?


(gdb) print &i
$7 = (int *) 0xbffff7bc
(gdb) print &b[i]
$8 = (int *) 0xbffff7bc
(gdb)

Yup, they are at the same address. Why? Doh! An array indexing error! b[10] is not within the array bounds! The last element is b[9].


Line 11 should use < instead of <= and read


for(int i=0; i <  SIZE; i++) {

Case closed.


An ugly observation is that programs like this bug out silently and "work" if you change the program to be initialized to 100 (or something bigger than SIZE).

Tic-tac-toe in python

Represent a board with a nine character string with spaces, "X", and "O". The rows of the board are concatenated to make the string, so the board


 X |   |   
---+---+---
   | O |   
---+---+---
   | X | O 


is the string "X   O  XO".

Given a board string, the function tic() returns the next board.  However, if the opponent can win (provided he makes the right moves), then tic() doesn't try to fight back, and may return bad moves.  A smarter program might continue to fight back as long as the game is not actually over.

fredag 26 november 2010

kontrollsiffra personnummer oneliner in python

def pnr(nr,d=2,c=0,i=0): return (i==9) and 10-(c%10) or pnr(str(nr),3-d,c+(d*int(nr[i]))%10+(d*int(nr[i]))/10,i+1)



def pnr(p):
    a,p=[0,2,4,6,8,1,3,5,7,9],[int(i) for i in p]
    return 10-(a[p[0]]+p[1]+a[p[2]]+p[3]+a[p[4]]+p[5]+a[p[6]]+p[7]+a[p[8]])%10



print "sista siffran = %d" % pnr("6408233234"[:-1])

torsdag 25 november 2010

Setup IPSec between two hosts

# based on http://www.ipsec-howto.org/x304.html


key() ( bits=$1; dd if=/dev/random count=$((bits/8)) bs=1| xxd -ps )

host1=2001:16e8:ff4e::1
host2=2001:16e7:cd3c::223:77ff:fe18:1421

cat > setkey.conf <
#!/usr/sbin/setkey -f

# Configuration for $host1

# Flush the SAD and SPD
flush;
spdflush;

# set the authentication header
# AH SAs using 128 bit long keys
add $host2 $host1 ah 0x200 -A hmac-md5  0x$(key 128);
add $host1 $host2 ah 0x300 -A hmac-md5  0x$(key 128);

# ESP SAs using 192 bit long keys (168 + 24 parity)
add $host2 $host1 esp 0x201 -E 3des-cbc  0x$(key 192);
add $host1 $host2 esp 0x301 -E 3des-cbc  0x$(key 192);

# Security policies
spdadd $host2 $host1 any -P out ipsec
           esp/transport//require
           ah/transport//require;

spdadd $host1 $host2 any -P in ipsec
           esp/transport//require
           ah/transport//require;

EOF

sudo setkey -f setkey.conf

# copy setkey.conf to the other computer (host2),
# and on the other computer swap the '-P in' and '-P out'
# in the spdadd and run the sudo setkey -f setkey.conf
# WARNING! You will not be able to contact the other
# computer if only one of them has enabled the IPSec rules
# with setkey.

måndag 1 november 2010

multiple users on one screen session

I got this info from here.

Suppose larsr and bill have accounts on a machine stella.
Thanks to 'screen' they can easily share a terminal window.

both log in to stella (for instace with ssh).

larsr starts screen

  • screen

and enables multiuser mode

  • CTRL-a :multiuser on

and permit bill to connect

  • CTRL-a :acladd bill


Now bill can connect to the screen by typing

  • screen -x larsr/

Note the '/'. If larsr has many screens, the screen id is entered after the '/'.

torsdag 21 oktober 2010


Building the LLVM based binary translator.


Lars.Rasmusson@sics.se 2010-10-21


1. Create clean build root file system


To make sure that we know what software we are using to build, we first create an entire root file system in a catalog. It containts its own build tools, independent of what the host OS itself uses.
Use debootstrap to install a bare bones root file system from a local repository:


sudo -s
BUILDROOT=$(pwd)/xenbuild
mkdir $BUILDROOT
debootstrap --variant=buildd --arch i386 lucid $BUILDROOT http://se.archive.ubuntu.com/ubuntu

2. chroot into it, configure and install build tools and git


Set time zone, and use apt to install the build tools


chroot $BUILDROOT
export LC_ALL=C
hostname builder
dpkg-reconfigure tzdata
adduser --disabled-password larsr

Use a local repository and install a lot of packets


echo > /etc/apt/sources.list deb http://se.archive.ubuntu.com/ubuntu lucid main restricted universe
apt-get update
apt-get install -y wget nano mercurial gawk openssh-client libz-dev \
  gettext libssl-dev libx11-dev bin86 bcc libncurses5-dev python-dev \
  texinfo bridge-utils uuid-dev git-core man iasl m4 flex bison cmake

3. check out xen from git on mac (edit on mac?)


Initialize my git user name


su larsr
cd $HOME
cat >> .gitconfig <<EOF
[user]
        name = Lars Rasmusson
        email = Lars.Rasmusson@sics.se
EOF
 
git clone ssh://lra@abakus.sics6.se/home/lra/src/xen-4.0-testing.git

4. Building and installing the Xen kernel


NOTE! These steps are only necessary if you are not already running xen on your testing machine.


4.1 Build


Prepare a new branch llvm-domain and check out a specific version of Xen. In the buildroot tree, do


cd ~/xen-4.0-testing
git checkout -b llvm-domain RELEASE-4.0.1
git clean -fd
make -j8 xen
make -j8 tools

4.2 Install


In the real root tree, copy the new xen-image to the /boot directory, and configure the bootloader (grub/grub2/whatever) to load the newly compiled xen.


First pack the xen files into a tar file and then into a deb. It makes it easier to remove or update later.


tar -czf xen.tgz -C dist/install/ .
fakeroot alien --to-deb xen.tgz

Then install the files in the real root file system. The update-rc.d commands makes xend and xendomains start when the system boots.


dpkg -i /home/larsr/xenbuild/home/larsr/xen-4.0-testing/xen_1-2_all.deb 
update-rc.d xend defaults
update-rc.d xendomains defaults

I'm running Ubuntu 10.04 Lucid Lynx which uses the grub2 boot loader. I installed Xen in grub by putting the following into /etc/grub.d/35_xen


#!/bin/sh
exec tail -n +3 $0
# This file is an example on how to add custom entries
 
menuentry "Xen 4.0 / Ubuntu 10.04 kernel 2.6.31.13 pvops" {
    insmod ext2
    set root=(hd0,1)
    multiboot (hd0,1)/xen-4.0.gz com1=115200,8n1 console=com1,vga
    module (hd0,1)/vmlinuz-2.6.31.13 console=tty0 console=hvc0 root=/dev/mapper/cluster5-root ro
    module (hd0,1)/initrd.img-2.6.31.13
    savedefault
}

and then run


update-grub

A linux running in Xen gets its console on hvc0, and for Ubuntu to start a getty on hvc0, put hvc0.conf in /etc/init/hvc0.conf


# hvc0 - getty
#
# This service maintains a getty on Xen's ttyhvc0 from the point the system is
# started until it is shut down again.
 
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]
 
respawn
exec /sbin/getty -8 115200 hvc0

Now you should be able to reboot and select the xen kernel from grub's boot menu. When you have booted, verify with


xm info

that Xen and xend are running.


6. build stub domain


This is done in the chrooted xenbuild directory.
First we build a stub domain, which makes the Xen Makefiles pull in several packages such as lwip, newlib, etc. and then compile mini-os which uses the packages.


D=~/xen-4.0-testing/stubdom
 
cd $D
make c-stubdom

Now the mini-os object files are compiled. We put them for convenience into a library


MO=$(pwd)/mini-os-x86_32-c
ar cr $D/../extras/minios.a  $MO/{blkfront,events,fbfront,fs-front,gntmap,gnttab,hypervisor,kernel,lock,main,mm,netfront,pcifront,sched}.o \
$MO/lib/{ctype,math,printf,stack_chk_fail,string,sys,xmalloc,xs}.o $MO/xenbus/xenbus.o $MO/console/{console,xencons_ring}.o

Now we are ready to build the cross g++ for minios


9. Building a cross compiler


You can compile some c++ programs for minios using the system's normal gcc, but it can't use iostreams etc, because the header files for g++ depend on the linux header files, and will break when compiling for minios.


We need a special c++ compiler that uses the right system include files, and newlib instead of glibc, or else
include iostream will not work. A g++ compiled for linux relies on some definitions
in the ctype.h file. Those definitions do not exist in minios, and therefore the iostream header files
generated for a linux g++ will not work on a minios system.


To build a g++ for minios, the sysroot of mini os must exist.

It got created (above) when we did make c-stubdom in the stubdom directory in the xen repository.
The minios sysroot is placed in $HOME/xen-4.0-testing/stubdom/cross-root-i686


# build tool chain
TARGET=i686-xen-elf
INSTALL=$HOME/sw
GCC_VER=4.5.0
###
 
export PATH=$INSTALL/bin:$PATH
 
mkdir -p $INSTALL/build
cd $INSTALL/build
 
(
wget http://ftp.sunet.se/pub/gnu/gmp/gmp-5.0.1.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/mpfr/mpfr-2.4.2.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gcc/infrastructure/mpc-0.8.1.tar.gz &
wait
)
 
tar -xjf gmp-5.0.1.tar.bz2
tar -xjf mpfr-2.4.2.tar.bz2
tar -xzf mpc-0.8.1.tar.gz
 
(
wget http://ftp.sunet.se/pub/gnu/binutils/binutils-2.20.tar.bz2 & 
wget http://ftp.sunet.se/pub/gnu/gcc/releases/gcc-$GCC_VER/gcc-core-$GCC_VER.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gcc/releases/gcc-$GCC_VER/gcc-g++-$GCC_VER.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gdb/gdb-7.1.tar.bz2 &
wait
)
tar -xjf binutils-2.20.tar.bz2
tar -xjf gcc-core-$GCC_VER.tar.bz2
tar -xjf gcc-g++-$GCC_VER.tar.bz2
tar -xjf gdb-7.1.tar.bz2
 
wget ftp://sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz
tar -xzf newlib-1.18.0.tar.gz
 
mkdir gmp-5.0.1/build; cd gmp-5.0.1/build
../configure --prefix=$INSTALL/build/gmp --disable-shared --enable-static
make -j8 && make install
cd ../../
mkdir mpfr-2.4.2/build; cd mpfr-2.4.2/build
../configure --prefix=$INSTALL/build/mpfr --with-gmp=$INSTALL/build/gmp \
   --disable-shared --enable-static
make -j8 && make install
cd ../../
mkdir mpc-0.8.1/build; cd mpc-0.8.1/build
../configure --prefix=$INSTALL/build/mpc --with-gmp=$INSTALL/build/gmp \
   --with-mpfr=$INSTALL/build/mpfr --disable-shared --enable-static
make -j8 && make install
cd ../../
 
patch -l -p0 <<EOF
--- binutils-2.20/gas/as.h      2010-06-15 18:49:19.771279712 +0000
+++ binutils-2.20/gas/as.h      2010-06-15 18:50:10.400278790 +0000
@@ -238,7 +238,7 @@
 #define know(p) gas_assert(p)  /* Verify our assumptions!  */
 #endif /* not yet defined */
 #else
-#define know(p)                        /* know() checks are no-op.ed  */
+#define know(p)        do {} while(0)  /* know() checks are no-op.ed  */
 #endif
 
 /* input_scrub.c */
EOF
 
cd $INSTALL/build
mkdir binutils-2.20/build; cd binutils-2.20/build
../configure \
   --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --enable-multilib --disable-nls --disable-shared --disable-threads \
   --with-sysroot=$HOME/xen-4.0-testing/stubdom/cross-root-i686/i686-xen-elf \
   --with-gcc --with-gnu-as --with-gnu-ld
make -j8 && make install
cd ../../
 
mkdir gcc-$GCC_VER/build; cd gcc-$GCC_VER/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --disable-multilib --enable-languages=c --with-newlib --disable-nls \
   --disable-shared --disable-threads --with-gnu-as --with-gnu-ld \
   --with-gmp=$INSTALL/build/gmp --with-mpfr=$INSTALL/build/mpfr \
   --with-mpc=$INSTALL/build/mpc \
   --without-headers
make -j8 all-gcc && make install-gcc
cd ../../
 
mkdir newlib-1.18.0/build; cd newlib-1.18.0/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --enable-multilib --with-gnu-as --with-gnu-ld --disable-nls 
make -j8 && make install
cd ../../
 
cd gcc-$GCC_VER/build
rm -rf *
../configure \
   --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --disable-multilib --enable-languages=c,c++ --with-newlib --disable-nls \
   --disable-shared --disable-threads --with-gnu-as --with-gnu-ld \
   --with-gmp=$INSTALL/build/gmp --with-mpfr=$INSTALL/build/mpfr \
   --with-mpc=$INSTALL/build/mpc \
   --with-sysroot=$HOME/xen-4.0-testing/stubdom/cross-root-i686/i686-xen-elf 
make -j8 && make install
cd ../../

Now the cross compiler and build tools for mini-os are installed in ~/sw.


9.1 Building a stub domain with the new g++ compiler


Here we assume that there are two c++ files in stubdom/c called main.cpp and mi.cpp.


We use the g++ and ld compiled for mini os.


GPP=i686-xen-elf-g++
LD=i686-xen-elf-ld

Compile the source files


cd $D
$GPP c/main.cpp -c -o c/main.o
$GPP c/mi.cpp -c -o c/mi.o

Link with the mini os object files and our own libstdc++.


$LD -r -d \
-m elf_i386 \
$D/mini-os-x86_32-c/arch/x86/x86_32.o \
-\( \
$D/c/main.o \
$D/c/mi.o \
$D/../extras/mini-os/app.lds \
--undefined main \
-\) \
-whole-archive \
$D/../extras/minios.a \
-no-whole-archive \
$D/mini-os-x86_32-c/lwip.a \
$D/libxc-x86_32/libxenguest.a \
$D/libxc-x86_32/libxenctrl.a \
$D/mini-os-x86_32-c/arch/x86/libx86_32.a \
-L$D/cross-root-i686/i686-xen-elf/lib \
-lstdc++ \
-lc -lm -lz -lpci \
-L$D/cross-root-i686/lib/gcc/i686-xen-elf/4.5.0 \
-lgcc \
-o $D/mini-os-x86_32-c/mini-os.o

Create the final executable by placing the code segments where they should be and pack the file


$LD \
-m elf_i386 \
$D/mini-os-x86_32-c/mini-os.o  \
-T $D/../extras/mini-os/arch/x86/minios-x86_32.lds \
-o $D/mini-os-x86_32-c/mini-os
 
gzip -f -5 -c $D/mini-os-x86_32-c/mini-os >$D/mini-os-x86_32-c/mini-os.gz

Voila! It should run on xen (start it from dom0, not from inside the buildroot).


xm create /dev/null  name=minios disk=file:/home/larsr/disk2.img,/dev/sda,r vif="" kernel=/home/larsr/xenbuild/home/larsr/xen-4.0-testing/stubdom/mini-os-x86_32-c/mini-os.gz -c

10. Build LLVM for minios.


You have to use a g++ and gcc that is compiled for minios. To use LLVM inside the minios, we must have the c++ header files in the compilation paths, and the correct libraries included.


cd ~
git clone ssh://lra@abakus.sics6.se/home/lra/src/llvm.git llvm.git
wget http://www.sics.se/~lra/lars-llvm.patch
wget http://www.sics.se/~lra/include-llvm-Config-config.h.patch
cd llvm.git/
git checkout 27ff7ebce9a8d5dd2631156d910fd4898a028dbb
patch -p1 <../lars-llvm.patch 
 
mkdir build
cd build
../configure --prefix=$INSTALL --enable-targets=x86,x86_64 --disable-threads --target=i686-xen-elf
 
make BUILD_DIRS_ONLY=1 -j8
 
sed -i -e"s@int\t_EXFUN(getpagesize@size_t\t_EXFUN(getpagesize@" \
  $INSTALL/i686-xen-elf/include/sys/unistd.h
 
patch include/llvm/Config/config.h ../../include-llvm-Config-config.h.patch
# minios has MAP_ANON but not MAP_ANONYMOUS and no TEMP file support
sed -i -e"s@#define HAVE_MMAP_ANONYMOUS 1@//#define HAVE_MMAP_ANONYMOUS 1@" include/llvm/Config/config.h
sed -i -e"s@#define HAVE_MKDTEMP 1@//#define HAVE_MKDTEMP 1@" include/llvm/Config/config.h
sed -i -e"s@#define HAVE_MKSTEMP 1@//#define HAVE_MKSTEMP 1@" include/llvm/Config/config.h
sed -i -e"s@#define HAVE_MKTEMP 1@//#define HAVE_MKTEMP 1@" include/llvm/Config/config.h
 
make VERBOSE=1 CXX="i686-xen-elf-g++" CC="i686-xen-elf-gcc"  -j8 -C lib -k
# use `-k` because the dynamic linking example 'lib/Transforms/Hello' will fail because -ldl is not supported
 
rm Debug/lib/libLLVM{Support,System}.a
cd lib/Support
rm -rf Debug
make VERBOSE=1 CXX="i686-xen-elf-g++" CC="i686-xen-elf-gcc" -j8
cd ../..
 
cd lib/System
rm -rf Debug
make VERBOSE=1 CXX="i686-xen-elf-g++ -isystem ${D:?}/../extras/mini-os/include/posix" CC="i686-xen-elf-gcc"  -j8
cd ../..

We should also build the llvm tools to get llvm-config


make -C tools -j8

This breaks when it compiles llvm-mc, but llvm-config is compiled ok.


Now llvm-config is in build/Debug/bin/llvm-config


11. Building an LLVM program for minios


The following has to be defined in some c source file, because newlib doesn't provide
lseek and fstat correctly after the xen patches (which renames them).
The domain crashes if linked with crt{begin,end}.o so instead
declare the missing variable __dso_handle.
The code below could be placed in main.cpp, for instance.


extern off_t lseek64(int a, off_t b, int c);
off_t lseek(int a, off_t b, int c) {
    return lseek64 (a, b, c);
}
 
int fstat64(int a, struct stat* b);
int fstat(int a, struct stat* b) {
    return fstat64(a,b);
}
 
void *__dso_handle = 0;

12. To compile it


Suppose the program is in $D/c/jit-test.o and that llvm-config is in the path.


cd $D
 
i686-xen-elf-g++ $(~/llvm.git/build/Debug/bin/llvm-config --cflags) -c c/jit-test.cpp -o c/jit-test.o
 
i686-xen-elf-ld -r -d -m elf_i386 $D/mini-os-x86_32-c/arch/x86/x86_32.o -\( \
 $D/c/jit-test.o \
 $D/../extras/mini-os/app.lds --undefined main -\) \
 -whole-archive $D/../extras/minios.a -no-whole-archive \
 $D/mini-os-x86_32-c/lwip.a $D/libxc-x86_32/libxenguest.a $D/libxc-x86_32/libxenctrl.a \
 $D/mini-os-x86_32-c/arch/x86/libx86_32.a \
 -L/home/larsr/llvm.git/build/Debug/lib \
 -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86AsmPrinter -lLLVMX86CodeGen -lLLVMSelectionDAG \
 -lLLVMAsmPrinter -lLLVMMCParser -lLLVMX86Info -lLLVMJIT -lLLVMExecutionEngine -lLLVMCodeGen -lLLVMScalarOpts \
 -lLLVMInstCombine -lLLVMTransformUtils -lLLVMipa -lLLVMAnalysis -lLLVMTarget -lLLVMMC -lLLVMCore \
 -lLLVMSupport -lLLVMSystem \
 -L$INSTALL/i686-xen-elf/lib  -lstdc++ \
 -L$D/cross-root-i686/i686-xen-elf/lib -lc -lm -lz -lpci  \
 -L$INSTALL/lib/gcc/i686-xen-elf/4.5.0  -lgcc \
 -o $D/mini-os-x86_32-c/mini-os.o
 
 
i686-xen-elf-ld \
-m elf_i386 \
$D/mini-os-x86_32-c/mini-os.o  \
-T /home/larsr/xen-4.0-testing/extras/mini-os/arch/x86/minios-x86_32.lds \
-o $D/mini-os-x86_32-c/mini-os
 
gzip -f -5 -c $D/mini-os-x86_32-c/mini-os >$D/mini-os-x86_32-c/mini-os.gz

13. Run the mini-os instance


From the dom0, as root, type


sudo xm create /dev/null kernel=/home/larsr/xenbuild/home/larsr/xen-4.0-testing/stubdom/mini-os-x86_32-c/mini-os.gz name=minios disk=file:/home/larsr/disk2.img,/dev/sda,r vif="" -c

tisdag 28 september 2010

otool and c++ symbol demangling

On Mac OS X, otool cannot demangle c++ symbols. For example

otool -SV Debug/lib/libLLVMX86Disassembler.a

spits out entries like

Archive : Debug/lib/libLLVMX86Disassembler.a
Table of contents from: Debug/lib/libLLVMX86Disassembler.a(__.SYMDEF SORTED)
size of ranlib structures: 488 (number 61)
size of strings: 2968
object           symbol name
X86Disassembler.o _LLVMInitializeX86Disassembler
X86Disassembler.o __ZN4llvm11raw_ostreamlsENS_9StringRefE
X86Disassembler.o __ZN4llvm11raw_ostreamlsEPKc
X86Disassembler.o __ZN4llvm11raw_ostreamlsEj
X86Disassembler.o __ZN4llvm14MCDisassemblerC2Ev
X86Disassembler.o __ZN4llvm14TargetRegistry22RegisterMCDisassemblerERNS_6TargetEPFPNS_14MCDisassemblerERKS1_E
X86Disassembler.o __ZN4llvm15SmallVectorImplINS_9MCOperandEE9push_backERKS1_
X86Disassembler.o __ZN4llvm15X86Disassembler18X86_32DisassemblerC1Ev

Fortunately, Mac OS X Developer tools comes with the GNU program c++filt, so we can use that to demangle the symbols



otool -SV Debug/lib/libLLVMX86Disassembler.a | awk 'NF==2 {printf "%s ",$1; system("c++filt " $2)} NF!=2{print $0}'


which looks much more readable

Archive : Debug/lib/libLLVMX86Disassembler.a
Table of contents from: Debug/lib/libLLVMX86Disassembler.a(__.SYMDEF SORTED)
size of ranlib structures: 488 (number 61)
size of strings: 2968
object           symbol name
X86Disassembler.o _LLVMInitializeX86Disassembler
X86Disassembler.o llvm::raw_ostream::operator<<(llvm::StringRef)
X86Disassembler.o llvm::raw_ostream::operator<<(char const*)
X86Disassembler.o llvm::raw_ostream::operator<<(unsigned int)
X86Disassembler.o llvm::MCDisassembler::MCDisassembler()
X86Disassembler.o llvm::TargetRegistry::RegisterMCDisassembler(llvm::Target&, llvm::MCDisassembler* (*)(llvm::Target const&))
X86Disassembler.o llvm::SmallVectorImpl::push_back(llvm::MCOperand const&)
X86Disassembler.o llvm::X86Disassembler::X86_32Disassembler::X86_32Disassembler()

fredag 18 juni 2010

Compiling a C++ xen mini-os

# build newlib, minios, the pci and xen libraries.
cd /root/xen/xen-4.0-testing.hg/stubdom/c
make c-stubdom

# put all the minios object files into an archive
MO=/root/xen/xen-4.0-testing.hg/stubdom/mini-os-x86_32-c

ar cr /root/xen/xen-4.0-testing.hg/extras/mini-os/minios.a \
 $MO/blkfront.o $MO/events.o $MO/fbfront.o $MO/fs-front.o $MO/gntmap.o \
 $MO/gnttab.o $MO/hypervisor.o $MO/kernel.o $MO/lock.o $MO/main.o \
 $MO/mm.o $MO/netfront.o $MO/pcifront.o $MO/sched.o $MO/lib/ctype.o \
 $MO/lib/math.o $MO/lib/printf.o $MO/lib/stack_chk_fail.o \
 $MO/lib/string.o $MO/lib/sys.o $MO/lib/xmalloc.o $MO/lib/xs.o \
 $MO/xenbus/xenbus.o $MO/console/console.o $MO/console/xencons_ring.o 

############################

D=/root/xen/xen-4.0-testing.hg/stubdom

GPP=$D/cross-root-i686/bin/i686-xen-elf-g++
LD=$D/cross-root-i686/bin/i686-xen-elf-ld

$GPP c/main.cpp -c -o c/main.o
$GPP c/mi.cpp -c -o c/mi.o

$LD -r -d \
-m elf_i386 \
/root/xen/xen-4.0-testing.hg/stubdom/mini-os-x86_32-c/arch/x86/x86_32.o \
-\( \
/root/xen/xen-4.0-testing.hg/stubdom/c/main.o \
/root/xen/xen-4.0-testing.hg/stubdom/c/mi.o \
/root/xen/xen-4.0-testing.hg/extras/mini-os/app.lds \
--undefined main \
-\) \
-whole-archive \
/root/xen/xen-4.0-testing.hg/extras/mini-os/minios.a \
-no-whole-archive \
/root/xen/xen-4.0-testing.hg/stubdom/mini-os-x86_32-c/lwip.a \
/root/xen/xen-4.0-testing.hg/stubdom/libxc-x86_32/libxenguest.a \
/root/xen/xen-4.0-testing.hg/stubdom/libxc-x86_32/libxenctrl.a \
/root/xen/xen-4.0-testing.hg/stubdom/mini-os-x86_32-c/arch/x86/libx86_32.a \
-L$D/cross-root-i686/i686-xen-elf/lib \
 -lstdc++ \
 -lc -lm -lz -lpci \
 -L$D/cross-root-i686/lib/gcc/i686-xen-elf/4.5.0 \
 -lgcc \
-o /root/xen/xen-4.0-testing.hg/stubdom/mini-os-x86_32-c/mini-os.o

# move around code in .o files into their correct sections
$LD \
-m elf_i386 \
$D/mini-os-x86_32-c/mini-os.o  \
-T /root/xen/xen-4.0-testing.hg/extras/mini-os/arch/x86/minios-x86_32.lds \
-o $D/mini-os-x86_32-c/mini-os

gzip -f -9 -c $D/mini-os-x86_32-c/mini-os >$D/mini-os-x86_32-c/mini-os.gz

######
##  
##  The following has to be defined because newlib doesn't provide
##  lseek and fstat correctly after the xen patches (which renames them).
##
##  The domain crashes if linked with crt{begin,end}.o so instead
##  declare the missing variable __dso_handle.
##
  extern off_t lseek64(int a, off_t b, int c);
  off_t lseek(int a, off_t b, int c) {
    return lseek64 (a, b, c);
  }

  int fstat64(int a, struct stat* b);
  int fstat(int a, struct stat* b) {
    return fstat64(a,b);
  }

  void *__dso_handle = 0;

### 
#  I was using a cross compiler with a special libstdc++, built with the commands
#  below.  Note that the directories here are local to my machine.

cd /root/xen/xen-4.0-testing.hg/stubdom/build/binutils
/home/larsr/test/i486-none-elf/build/binutils-2.20/configure \
   --prefix=/root/xen/xen-4.0-testing.hg/stubdom/cross-root-i686 \
   --target=i686-xen-elf --enable-interwork \
   --enable-multilib --disable-nls --disable-shared --disable-threads \
   --with-sysroot=/root/xen/xen-4.0-testing.hg/stubdom/cross-root-i686/i686-xen-elf
   --with-gcc --with-gnu-as --with-gnu-ld
make -j8 && make install


cd /root/xen/xen-4.0-testing.hg/stubdom/build/gcc
/home/larsr/test/i486-none-elf/build/gcc-4.5.0/configure \
   --prefix=/root/xen/xen-4.0-testing.hg/stubdom/cross-root-i686 \
   --target=i686-xen-elf --enable-interwork \
   --disable-multilib --enable-languages=c,c++ --with-newlib --disable-nls \
   --disable-shared --disable-threads --with-gnu-as --with-gnu-ld \
   --with-gmp=/home/larsr/test/i486-none-elf/build/gmp \
   --with-mpfr=/home/larsr/test/i486-none-elf/build/mpfr \
   --with-mpc=/home/larsr/test/i486-none-elf/build/mpc \
   --with-sysroot=/root/xen/xen-4.0-testing.hg/extras/mini-os/include
make -j8 && make install
cd ../../

tisdag 15 juni 2010

Building a gcc cross compiler

This was based on an example from http://www.arklyffe.com/main/content/arm-gcc-toolchain-build, but I had to use --disable-multilib to get gcc to compile. Or else, it would fail some of its ./configuration steps (I think for instance libz failed).

Note that gcc is built twice, once to get a boot strap compiler that can compile newlib for the target platform, and then again to compile gcc to use newlib as its C library.

# build tool chain
TARGET=arm-none-eabi
TARGET=i486-none-elf
INSTALL=$HOME/test/$TARGET
GCC_VER=4.5.0
###

export PATH=$INSTALL/bin:$PATH

mkdir $INSTALL
cd $INSTALL
mkdir build
cd build

(
wget http://ftp.sunet.se/pub/gnu/gmp/gmp-5.0.1.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/mpfr/mpfr-2.4.2.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gcc/infrastructure/mpc-0.8.1.tar.gz &
wait
)

tar -xjf gmp-5.0.1.tar.bz2
tar -xjf mpfr-2.4.2.tar.bz2
tar -xzf mpc-0.8.1.tar.gz

(
wget http://ftp.sunet.se/pub/gnu/binutils/binutils-2.20.tar.bz2 & 
wget http://ftp.sunet.se/pub/gnu/gcc/releases/gcc-$GCC_VER/gcc-core-$GCC_VER.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gcc/releases/gcc-$GCC_VER/gcc-g++-$GCC_VER.tar.bz2 &
wget http://ftp.sunet.se/pub/gnu/gdb/gdb-7.1.tar.bz2 &
wait
)
tar -xjf binutils-2.20.tar.bz2
tar -xjf gcc-core-$GCC_VER.tar.bz2
tar -xjf gcc-g++-$GCC_VER.tar.bz2
tar -xjf gdb-7.1.tar.bz2

wget ftp://sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz
tar -xzf newlib-1.18.0.tar.gz

mkdir gmp-5.0.1/build; cd gmp-5.0.1/build
../configure --prefix=$INSTALL/build/gmp --disable-shared --enable-static
make -j8 && make install
cd ../../
mkdir mpfr-2.4.2/build; cd mpfr-2.4.2/build
../configure --prefix=$INSTALL/build/mpfr --with-gmp=$INSTALL/build/gmp \
   --disable-shared --enable-static
make -j8 && make install
cd ../../
mkdir mpc-0.8.1/build; cd mpc-0.8.1/build
../configure --prefix=$INSTALL/build/mpc --with-gmp=$INSTALL/build/gmp \
   --with-mpfr=$INSTALL/build/mpfr --disable-shared --enable-static
make -j8 && make install
cd ../../

patch -l -p0 <<EOF
--- binutils-2.20/gas/as.h      2010-06-15 18:49:19.771279712 +0000
+++ binutils-2.20/gas/as.h      2010-06-15 18:50:10.400278790 +0000
@@ -238,7 +238,7 @@
 #define know(p) gas_assert(p)  /* Verify our assumptions!  */
 #endif /* not yet defined */
 #else
-#define know(p)                        /* know() checks are no-op.ed  */
+#define know(p)        do {} while(0)  /* know() checks are no-op.ed  */
 #endif
 
 /* input_scrub.c */
EOF

mkdir binutils-2.20/build; cd binutils-2.20/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --enable-multilib --disable-nls --disable-shared --disable-threads \
   --with-gcc --with-gnu-as --with-gnu-ld
make -j8 && make install
cd ../../

mkdir gcc-$GCC_VER/build; cd gcc-$GCC_VER/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --disable-multilib --enable-languages=c --with-newlib --disable-nls \
   --disable-shared --disable-threads --with-gnu-as --with-gnu-ld \
   --with-gmp=$INSTALL/build/gmp --with-mpfr=$INSTALL/build/mpfr \
   --with-mpc=$INSTALL/build/mpc --without-headers
make -j8 all-gcc && make install-gcc
cd ../../

mkdir newlib-1.18.0/build; cd newlib-1.18.0/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --enable-multilib --with-gnu-as --with-gnu-ld --disable-nls 
make -j8 && make install
cd ../../

rm -rf gcc-$GCC_VER/build
mkdir gcc-$GCC_VER/build; cd gcc-$GCC_VER/build
../configure --prefix=$INSTALL --target=$TARGET --enable-interwork \
   --disable-multilib --enable-languages=c,c++ --with-newlib --disable-nls \
   --disable-shared --disable-threads --with-gnu-as --with-gnu-ld \
   --with-gmp=$INSTALL/build/gmp --with-mpfr=$INSTALL/build/mpfr \
   --with-mpc=$INSTALL/build/mpc
make -j8 && make install
cd ../../

tisdag 1 juni 2010

Building umip on Ubuntu Lucid

umip is a mobile IPv6 daemon

apt-get install autoconf byacc flex indent
wget 'http://www.umip.org/gitweb?p=umip.git;a=snapshot;h=d1c240f3deb690af902ce1ff128780551ff6141c;sf=tgz'  -O umip.tgz
tar xfz umip.tgz
cd umip
autoreconf -fiv
./configure
make

torsdag 27 maj 2010

Setting up a compute cloud at Amazon

First you have to register an account to use Amazon's many Web Services.
"Sign Up", and enter email, name and address.

When that's done, sign up to the Amazon Elastic MapReduce service. Here you have to enter your credit card number, and address.  Then Amazon calls you and have you enter (or say!) a PIN code that is shown on the screen.

Then they will mail you to let you know that you have enrolled in the services Amazon Elastic MapReduce, Amazon Virtual Private Cloud, Amazon Elastic Compute Cloud, Amazon Simple Storage Service.

You are also told that you have now an access-key that is used in the Web Services interface to Amazon's services.

Now, lets see the Getting Started Guide.
We need to create a "bucket" to store data and results.  There are both command line tools, and a great firefox plugin Amazon S3 Firefox Organizer (S3Fox)

$ KEY="" SKEY="" # fill in values from Amazon
$ curl timkay.com/aws/aws -o aws; chmod a+x aws
$ echo $KEY >~/.awssecret; chmod go-rwx ~/.awssecret ; echo $SKEY >>~/.awssecret

$ ./aws mkdir sics-lra

tisdag 25 maj 2010

Creating a Xen Ubuntu Lucid DomU

# work in progress
# create a new file system, add a user, enable networking with dummy ip4 address,  create the VM

vg=cluster5
export vm=myvm
LUCID=/mnt
lvcreate --size 1G --name $vm  $vg
mkfs.ext4 /dev/$vg/$vm

mount /dev/$vg/$vm $LUCID
debootstrap --variant=buildd --arch i386 lucid $LUCID http://se.archive.ubuntu.com/ubuntu
chroot $LUCID
export LC_ALL=C

cat > /etc/apt/sources.list <<EOF
deb http://se.archive.ubuntu.com/ubuntu lucid          main restricted universe
deb http://se.archive.ubuntu.com/ubuntu lucid-security main restricted universe
EOF

apt-get update
apt-get -y install ubuntu-minimal openssh-server
adduser --ingroup sudo larsr
# fill in stuff

echo $vm > /etc/hostname
cat >>/etc/network/interfaces <<EOF
auto eth0
iface eth0 inet static
 address 1.1.1.1
 netmask 255.255.255.255
EOF

exit
umount $LUCID

xm create /dev/null kernel=/boot/vmlinuz-2.6.31.13 ramdisk=/boot/initrd.img-2.6.31.13 name=$vm  disk=phy:/dev/$vg/$vm,/dev/xvda1,w  root=/dev/xvda1 vif="mac=00:16:3e:0e:34:14,vifname=$vm" -c

Then you can create snapshots of the disk very quickly (thanks to LVM)
orig=vm
vm=newvm
lvcreate --snapshot --size 1G --name $vm $vg/$orig
mount /dev/$vg/$vm /mnt
echo $vm > /mnt/etc/hostname 
umount /mnt

onsdag 12 maj 2010

Quick and Dirty Flattr button to blogger.com - adding a HTML/Javascript Gadget

1. Submit your blog url to Flattr  https://flattr.com/submit
2. Copy the html code that is generated to your clip board (mark it and press CTRL-C, or CMD-C)
3. Go to your blogger dashboard at http://www.blogger.com/home. It will list all your blogs.
4. At the entry for your blog, click on "Settings" (or what it is called in your language), second-next to the blue "New Post" button.
5. Click on the "Layout" tab, and choose the "Page Elements" sub-tab
6. Click on "Add a Gadget" (you can choose above, on the side menu, or at the bottom.  I chose to put it in the site menu).
Add7. Click on the "HTML/JavaScript
" gadget.
8. Paste the flattr HTML code from step 2 into the content box, and add some title and explaining text if you like.
9. Click save.

Now this will show the Flattr button on every page of your blog.  View your blog to see that it actually works.

tisdag 11 maj 2010

building xen3.4.2 in a lucid root file system



# create the pristine root file system

LUCID=$HOME/lucid
mkdir -p $LUCID

wget http://se.archive.ubuntu.com/ubuntu/pool/main/d/debootstrap/debootstrap_1.0.20ubuntu1_all.deb
sudo dpkg -i debootstrap_1.0.20ubuntu1_all.deb
sudo debootstrap --variant=buildd --arch i386 lucid $LUCID http://se.archive.ubuntu.com/ubuntu


# go into it
sudo chroot $LUCID

export LC_ALL=C

echo > /etc/apt/sources.list deb http://se.archive.ubuntu.com/ubuntu lucid main restricted universe
apt-get update
apt-get install -y wget nano mercurial git-core gawk openssh-client libz-dev \
gettext libssl-dev libx11-dev bin86 bcc libncurses5-dev python-dev texinfo \
bridge-utils uuid-dev iasl

mkdir /root/xen
cd /root/xen

XENVER=3.4.2
XENVER=4.0.0
wget http://bits.xensource.com/oss-xen/release/$XENVER/xen-$XENVER.tar.gz
tar xfz xen-$XENVER.tar.gz
cd xen-$XENVER
make -j8 world

# now everything is in dist, install like this
cd dist
sudo sh install.sh
mkinitramfs -v -o /boot/initrd.img-2.6.18.8-xen 2.6.18.8-xen

# make a grub entry and reboot
# WARNING! make sure that the kernel can boot your file system 
# (LVM requires the dm module, etc.)

torsdag 6 maj 2010

Install Ubuntu Lucid Lynx root file system from command line

Here is a link to a more thorough post

If you are running an older version of ubuntu, or some other debian distribution, you can create a lucid lynx root file system like this

First install the debootstrap package (it should include the settings file for lucid lynx)
1. download it from http://packages.ubuntu.com/intrepid-backports/all/debootstrap/download
2. install it with sudo dpkg -i NAME_OF_PACKAGE.deb

Create the root filesystem (on a mounted partition, that you have prepared, here I just mkdir a place)

3. mkdir ~/lucid
4. sudo debootstrap --variant=buildd --arch i386 lucid ~/lucid http://archive.ubuntu.com/ubuntu/

That's it!

Debootstrap copies the /etc/passwd so  you can login, but it doesn't copy over the network settings or other servers, so make sure that they are correct before starting a machine with the new root file system.

fredag 23 april 2010

Send text to Mac ClipBoard from python (like pbcopy)

def sendToClipBoard(string):
    from AppKit import NSPasteboard,NSObject,NSStringPboardType
    pasteboard = NSPasteboard.generalPasteboard()
    emptyOwner = NSObject.alloc().init()
    pasteboard.declareTypes_owner_([NSStringPboardType], emptyOwner)
    pasteboard.setString_forType_(string, NSStringPboardType)

if __name__ == "__main__":
    import sys
    sendToClipBoard(sys.stdin.read())

Idea from this marvellous post about writing a SIMBL plugin in Python.

About the building the TabDump plugin:
When I built the SIMBL plugin, I had to compile it with
/usr/bin/python setup.py py2app
and I had to set MaxBundleVersion='6531' in setup.py

Anothere way is to build it with
/usr/bin/python setup.py py2app -s --site-packages
It doesn't copy in 16 MB python stuff into the bundle.

For some reason /opt/local/bin/python doesn't work, because
then py2app makes only a i368 binary, not a x86_64 binary.  Weird.

tisdag 13 april 2010

Installation log for Oja

* In Sharing: enabled Remote Login, Remote Management, File Sharing
* Installed 3Connect
* Configured WiFi network Oja
* Enabled Internet sharing of 3 HUAWEI modem to Airport
* Installed DynDNS updater
* made the system connect the modem at boot, not at login
crontab -e
with
@reboot /usr/sbin/networksetup -connectpppoeservice HUAWEIMobile-Modem

* Set the system to mount disks at boot, not only at login, and keep them mounted.

sudo defaults write /Library/Preferences/SystemConfiguration/autodiskmount AutomountDisksWithoutUserLogin -bool true

* Disabled the SIM PIN with the 3Connect tool. (If it's on it will prompt for the SIM after a power failure which prevents automatic reconnection.)

* in System Prefs, Energy Saver set Computer Sleep to never, and check "start up automatically after power failure", and at "schedule…" set "wake up every day" at 12:00

* added this entry to the root crontab to make computer reboot if the word reboot is written into the midnight_command file

# twenty minutes past midnight
20 0 * * * grep -q reboot /Users/larsr/midnight_command && /sbin/reboot -q
# one minute to half past, every hour
29 * * * * /Users/larsr/checknet.command

* Problem: pppd for internet sharing and pppd for the 3G modem are competing and killing each other! Probably the networksetup command is no good.

* changed back so that the user logs in automatically, The start of modem connection is now handled by the login procedure.

* Set the system to share internet connection from 3G modem to airport.

* added .profile and .bashrc files to set EDITOR=nano (for easier crontab editing)

* installed Xcode from the OS X dvd, and updated the software
* installed MacPorts
* installed gphoto2 from macports
* installed a newer version of libgphoto2 2.4.9 and gphoto2 2.4.9 because it supports D5000. I changed the version number and checksums in the portfile and added a patch that I got from libgphoto2's svn repository that added a check if scsi/sg was supported (which it isn't on OS X)

* to run gphoto2, the PTPCamera.app must not be running, but it is started automatically when the camera is plugged in, so I do "killall PTPCamera" before running gphoto2. However, that makes Sofortbild not run anymore.

* necessary power outlets: 1 for UPS, 1 for Computer, 1 for Disk, 1 for Backup Disk, 1 for Camera, 1 for heater, 1 for backup laptop. Don't put heater on the UPS.

* installed kermit. check receiving strength with kermit:
kermit -l /dev/tty.HUAWEIMobile-Pcui -C "set carrier-watch off,connect" | awk '/SSI/ {print $0}'
or issue the command "AT+CSQ"

* installed Sofortbild
* the logitech quickcam vision pro need a "UVC compatible" video program to record HD. With QuickTime it's only 640x400. No - actually QuickTime can record from the camera at HD quality, but the quality sucks and the framerate is really bad. Bummer.

* Ran Apple's "Image Capture" program, which can fetch pictures from the camera's card. And set the camera to Shared, so others on the lan can now fetch from the card too.

* installed the Nikon programs that came with the camera, but they were useless so I removed them

* installed MacPorts packages tuntaposx and aiccu, and configured the IPv6 tunnel by putting the config file in /opt/local/etc/aiccu.conf, and starting the services with the launchd commands
sudo launchctl load -w /Library/LaunchDaemons/org.macports.tuntaposx.plist
sudo launchctl load -w /Library/LaunchDaemons/org.macports.aiccu.plist
It remains to see what happens when the network connection is restarted - will launchd restore it?

* installed appscript into the default python installation with "sudo easy_install appscript"

* added a kill gphoto2 before the run of gphoto2 because it sometimes hangs.

* installed gpg,  imported a public key to encrypt photos with. gave it ultimate trust,  add encryption to the click-script.