Dreaming while programming

At work I encountered a situation when having a small standalone tool to display the variables with expanded types and when available also display the address. For a basic look at symbols and their addresses, the tool readelf can be used, the issue is that it does not expand the types and only shows the addresses of the main variable, not of its sub stuff in structs or arrays of stuff. Other tools which can be used and I am aware of are: dwarfdump, objdump and pahole. Each has its strong and weak points but I did not manage to find the feature I wanted.

Fantasy and dreams

Lets get to my dreamy part. The file format for the executables is called elf and the debug format mostly used with it is called dwarf, my fantasy comes from this little naming joke. I spent a lot of hour reading and rereading parts of the DWARF documentation to come up with a solution and good enough implementation and having dreams is a good motivation to do exactly that.

Implementation

As a language of choice I went with Python and the library pyelftools due to its qualities and portability. A very good starting point, inspiration and a great helping tool I used Dwarf Explorer (DWEX) by Seva Alekseyev. DWEX is an awesome tool to discover and roam around DWARF data, I can recommend it with confidence to anyone who wants to work with DWARF.

I used DWEX as a starting point to get pyelftools to read ELF file and its DWARF data, it provided a great jump at the beginning and I could focus on messing around with following the types and references. The whole thing is a bit of a mess and does multiple passes on the same data (I deem it inefficient) in an attempt to transfer DWARF references into a Python dictionary and than traverse it to make a flat version which I so desire.

Long story short, it was a headache but it works, there are some issues though. The main issue being that it uses a Godlt technique called recursion and will get into an infinite loop while forcing its way through references if the type contains a reference to either itself or different type which contain a reference to it. I intend to solve this issue in the future, but it is present at the moment.

There is one more thing to be aware of when trying to develop a tool like this, and that is that embedded world is a chaotic place and each vendor / company interprets and uses DWARF a tiny bit differently but since they also provide the whole toolchain you will not be aware of this mess until you try to write your own tool like this one. Stable DWARF version is a luxury, each vendor uses different and it is impossible to force their tools to generate a specific version. So the code of this little tool might need to be adjusted to work with multiple DWARF versions which in best case mean supporting multiple field names and in worst reworking whole routines, my headache.

Result

The result can be seen below. The only thing this script does is to print the whole list to stdout and let user handle the rest. It works best when paired with fzf but grep works fine as well. The whole idea was to get the whole name when searching for a member of some inner structure and I believe I managed to achieve this, somehow.

> python main.py ../Playground_C/build/playground_c
0x4030  my_stuff
0x4030  my_stuff.first
0x4030  my_stuff.first.number_1
0x4038  my_stuff.first.number_2
0x4040  my_stuff.second
0x4040  my_small_stuff
0x4040  my_small_stuff.one
0x4040  my_small_stuff.one[0]
...
0x4057  my_small_stuff.one[23]
0x4040  my_small_stuff.structure
0x4040  my_small_stuff.structure.first

Conclusion

The script can be found in my Gitea instance as Miscellaneous/elf_symbols where I also rumble in README about how I am tired of big proprietary tools.

PS: I believe this post is a mess, but well so be it. It is a good introduction to my fantasy world.

PS2: I wanted to write this half a year ago but I could not find a right words to do so but it was still stuck in my head…