Although often mistaken for "hacking", reversing is different in that it is a dignified science much as a biology researcher would seek to understand the working of natural processes. The goal is to discover information that was previously unavailable. For software, it means taking a peek into the guts of programs and understanding what they are really doing. On college campuses, this is the subject of entire courses. You don't have to go to college for this stuff though - a creative mind, code breaking skills, programming, and logical analysis abilities are much of what you will need.
Why reverse engineering? Apart from paranoia when programs arbitrarily run on my computer without my knowledge, more plausible reasons exist:
- Be able to create competing or better software products.
- Understand security = encryption, malicious software, product protections, audit programs.
- Interoperability with other [undocumented] software systems.
- Evaluate software quality and robustness.
You must be able to reverse at all levels of program complexity. System-level reversing involves understanding what the OS and applications are doing, before delving into code-level reversing where you deal with algorithms and design concepts. You should be able to work with native [C/C++] binaries (as opposed to [Java/.Net] bytecode - which requires a virtual machine to execute - and is easier).
You'll need an arsenal of tools to get your work done, including monitors, sniffers, and explorers (to show you information about the programs in the operating environment), disassembers (to provide textual representations of program binaries), debuggers (to observe software at runtime, including setting breakpoints and tracing through code), and decompilers (build source code from assembly/object code).
You'll certainly run into legal issues. There's really no need to reverse patented technologies (since they are publicly published and protected from duplication by law). Trade secrets can nevertheless be reversed, as long as they are available on the open market and obtained legally. The DMCA can seem threatening, but it really doesn't affect most reverse engineering.
The DMCA prohibits circumventing copyright protection systems (e.g defeating DRM) and creating technologies to circumvent copyright protection systems (e.g. key-gens). If you develop a license key generation program, you violate the second provision, and if you use it, you violate the first. You are however excused from the DMCA if your intent is for interoperability, research, security evaluation, educational institutions and library evaluation (presumably before purchase), government investigation, regulation (e.g. restrict usage by minors), and protection of privacy (e.g. remove features that transmit personal information).
Once you have grasped this high level view of reversing, you need patience and persistence to start getting good at it. The benefits of this skill are endless. The desire to know what's going on and to gain competitive advantage shall always be a primary motivator, but proficiency at this skills is what will guarantee how useful this world becomes to you. You'll end up with gains in observation skills and avid attention to detail. If you are a software developer, it'll change how you design and write programs. If you are a software tester, you'll be more productive at how you do testing and quality control.
