A Short Course: Compilers, Assemblers, Linkers, Loaders

Source and copy right @ https://courses.cs.washington.edu/courses/cse378/97au/help/compilation.html

Assembler: A computer will not understand any program written in a language, other than its machine language. The programs written in other languages must be translated into the machine language. Such translation is performed with the help of software. A program which translates an assembly language program into a machine language program is called an assembler. If an assembler which runs on a computer and produces the machine codes for the same computer then it is called self assembler or resident assembler. If an assembler that runs on a computer and produces the machine codes for other computer then it is called Cross Assembler.

Assemblers are further divided into two types: One Pass Assembler and Two Pass Assembler. One pass assembler is the assembler which assigns the memory addresses to the variables and translates the source code into machine code in the first pass simultaneously. A Two Pass Assembler is the assembler which reads the source code twice. In the first pass, it reads all the variables and assigns them memory addresses. In the second pass, it reads the source code and translates the code into object code.

Compiler: It is a program which translates a high level language program into a machine language program. A compiler is more intelligent than an assembler. It checks all kinds of limits, ranges, errors etc. But its program run time is more and occupies a larger part of the memory. It has slow speed. Because a compiler goes through the entire program and then translates the entire program into machine codes. If a compiler runs on a computer and produces the machine codes for the same computer then it is known as a self compiler or resident compiler. On the other hand, if a compiler runs on a computer and produces the machine codes for other computer then it is known as a cross compiler.

Interpreter: An interpreter is a program which translates statements of a program into machine code. It translates only one statement of the program at a time. It reads only one statement of program, translates it and executes it. Then it reads the next statement of the program again translates it and executes it. In this way it proceeds further till all the statements are translated and executed. On the other hand, a compiler goes through the entire program and then translates the entire program into machine codes. A compiler is 5 to 25 times faster than an interpreter.

By the compiler, the machine codes are saved permanently for future reference. On the other hand, the machine codes produced by interpreter are not saved. An interpreter is a small program as compared to compiler. It occupies less memory space, so it can be used in a smaller system which has limited memory space.

Linker: In high level languages, some built in header files or libraries are stored. These libraries are predefined and these contain basic functions which are essential for executing the program. These functions are linked to the libraries by a program called Linker. If linker does not find a library of a function then it informs to compiler and then compiler generates an error. The compiler automatically invokes the linker as the last step in compiling a program.

Not built in libraries, it also links the user defined functions to the user defined libraries. Usually a longer program is divided into smaller subprograms called modules. And these modules must be combined to execute the program. The process of combining the modules is done by the linker.

Loader: Loader is a program that loads machine codes of a program into the system memory. In Computing, a loader is the part of an Operating System that is responsible for loading programs. It is one of the essential stages in the process of starting a program. Because it places programs into memory and prepares them for execution. Loading a program involves reading the contents of executable file into memory.  Once loading is complete, the operating system starts the program by passing control to the loaded program code. All operating systems that support program loading have loaders. In many operating systems the loader is permanently resident in memory.

Compilers, Assemblers, Linkers, Loaders: A Short Course

This document briefly describes what happens when you compiler and run
a program. More details can be found in Compilers, Principles,
Techniques, and Tools
by Aho, Sethi, and Ullman (CSE 401 book)
and Appendix A of Computer Organization and Design by
Patterson and Hennesey (CSE 378 book).

Compiling a Program

When you type cc at the command line a lot of stuff happens.
There are four entities involved in the compilation process:
preprocessor, compiler, assembler, linker (see Figure 1).

 

The internals of cc

Figure 1: The internals of cc.

 

First, the C preprocessor cpp expands all those macros
definitions and include statements (and anything else that starts with
a #) and passes the result to the actual compiler. The
preprocessor is not so interesting because it just replaces some short
cuts you used in your code with more code. The output of cpp
is just C code; if you didn’t have any preprocessor statements in your
file, you wouldn’t need to run cpp. The preprocessor does
not require any knowledge about the target architecture. If you had
the correct include files, you could preprocess your C files on a
LINUX machine and take the output to the instructional machines and
pass that to cc. To see the output of a preprocessed file,
use cc -E.

The compiler effectively translates preprocessed C code into assembly
code, performing various optimizations along the way as well as
register allocation. Since a compiler generates assembly code
specific to a particular architecture, you cannot use the assembly
output of cc from an Intel Pentium machine on one of the
instructional machines (Digital Alpha machines). Compilers are very
interesting which is one of the reasons why the department offers an
entire course on compilers (CSE 401). To see the assembly code
produced by the compiler, use cc -S.

The assembly code generated by the compilation step is then passed to
the assembler which translates it into machine code; the resulting
file is called an object file. On the instructional machines, both
cc and gcc use the native assembler as that
is provided by UNIX. You could write an assembly language program and
pass it directly to as and even to cc (this is what
we do in project 2 with sys.s). An object file is a binary
representation of your program. The assembler gives a memory
location
to each variable and instruction; we will see later that
these memory locations are actually represented symbolically or via
offsets. It also make a lists of all the unresolved references that
presumably will be defined in other object file or libraries,
e.g. printf. A typical object file contains the
program text (instructions) and data (constants and strings),
information about instructions and data that depend on absolute
addresses, a symbol table of unresolved references, and possibly some
debugging information. The UNIX command nm allows you to
look at the symbols (both defined and unresolved) in an object file.

Since an object file will be linked with other object files and
libraries to produce a program, the assembler cannot assign absolute
memory locations to all the instructions and data in a file. Rather,
it writes some notes in the object file about how it assumed
things were layed out. It is the job of the linker to use these notes
to assign absolute memory locations to everything and resolve any
unresolved references. Again, both cc and gcc on
the instructional machines use the native linker, ld. Some
compilers chose to have their own linkers, so that optimizations can
be performed at link time; one such optimization is that of aligning
procedures on page boundaries. The linker produces a binary
executable that can be run from the command interface.

Notice that you could invoke each of the above steps by hand. Since
it is an annoyance to call each part separately as well as pass the
correct flags and files, cc does this for you. For example,
you could run the entire process by hand by invoking /lib/cpp
and then cc -S and then /bin/as and finally
ld. If you think this is easy, try compiling a simple
program in this way.

Running a Program

When you type a.out at the command line, a whole bunch of
things must happen before your program is actually run. The loader
magically does these things for you. On UNIX systems, the loader
creates a process. This involves reading the file and creating an
address space for the process. Page table entries for the
instructions, data and program stack are created and the register set
is initialized. Then the loader executes a jump instruction to the
first instruction in the program. This generally causes a page fault
and the first page of your instructions is brought into memory. On
some systems the loader is a little more interesting. For example, on
systems like Windows NT that provide support for dynamically loaded
libraries (DLLs), the loader must resolve references to such libraries
similar to the way a linker does.

Memory

Figure 2 illustrates a typical layout for program memory. It is the
job of the loader to map the program, static data (including globals
and strings) and the stack to physical addresses. Notice that the
stack is mapped to the high addresses and grows down and the program
and data are mapped to the low addresses. The area labeled
heap is where the data you allocate via malloc is
placed. A call to malloc may use the sbrk system
call to add more physical pages to the program’s address space (for
more information on malloc, free and sbrk,
see the man pages).

 

Memory layout

Figure 2: Memory layout.

 

Procedure Call Conventions

A call to a procedure is a context switch in your program. Just like
any other context switch, some state must be saved by the calling
procedure, or caller, so that when the called procedure, or
callee, returns the caller may continue execution without
distraction. To enable separate compilation, a compiler must follow a
set of rules for use of the registers when calling procedures. This
procedure call convention may be different across compilers
(does cc and gcc use the same calling convention?)
which is why object files created by one compiler cannot always be
linked with that of another compiler.

A typical calling convention involves action on the part of the caller
and the callee. The caller places the arguments to the callee in some
agreed upon place; this place is usually a few registers and the
extras are passed on the stack (the stack pointer may need to be
updated). Then the caller saves the value of any registers it will
need after the call and jumps to the callee’s first instruction. The
callee then allocates memory for its stack frame and saves any
registers who’s values are guaranteed to be unaltered through a
procedure call, e.g. return address. When the callee is
ready to return, it places the return value, if any, in a special
register and restores the callee-saved registers. It then pops the
stack frame and jumps to the return address.


Original version from CSE451, Autumn 1996.
Modified by wolman@cs.washington.edu, Autumn 1997.

The Best Month to Bid COE in Singapore

The best month to bid COE in Singapore is Feb, followed by Jan, and Dec. My prediction of the current COE price will reach a year low and then rebound to higher price. There will be a crash soon, maybe in 2 years, no one knows.

I contributed my full weekends to work out these four charts, starting to write Python and R, plus using the powerful yet not popular IBM Bluemix cloud services.

Singapore COE is truly the smartest real money collected by the Government. You can’t believe there was 2-dollar COE, and the highest price had reached more than 10K +.

The correlation between COE price and COE Quota is -0.5532, COE price and COE demand is  -0.5334, both low negative correlation, means that higher quota, much lower price, but higher demand, not lower price as supply is limited.

Values for bids and quota premium are calculated by average as two bids in the same month after 2002. Before 1999, Cat A is Cat 1 + Cat 2, and Cat B is Cat 3 + Cat 4.

After looking these two charts, I can only conclude that people in Singapore are really rich.

a

b

chart1

chart2

Web HTTP Streaming Research

Recently, I am searching for technologies about web streaming.

Streaming video/audio needs client side to receive data, and server side to prepare the data.

MDN: An Introduction to Live streaming web audio and video

HTTP Live Streaming

The Basics of HTTP Live Streaming

DASH Adaptive Streaming for HTML 5 Video

Apple Developer for Streaming

Streaming Data format Comparison

Open Leaders: DASH.js

Power geek: Video.js

Tools:

VLC Player

FFmpeg

GPAC

Online free video editor: Wevideo

Online free MPEG-DASH generator

The free tools are great.

You can also generate MPEG-2 TS format streaming file using VLC player.

It seems that DASH.JS is the only free JS API support HTTP live streaming in the client without any other plugin.

However, it can’t work in IE 11- and iOS.

HTTP Cookie

1. HTTP Cookie Explained
2. Cookie and Security

3. RFC 2109
4. Session Cookie
A session cookie or transient cookie is created by simply not setting a date in the Set-Cookie option when an application creates the cookie.
Session cookies are often used to enable a site to be able to track the pages that a user has visited during a visit so that information can be customized for the user in some way. Some sites use Secure Sockets Layer (SSL) to encrypt the information contained in a cookie.
5. Persistent Cookie
A persistent cookie, an expiration date is set and the cookie is stored on the user’s hard drive until the expiration date or until the user deletes it.
6. HTTP-Only Cookie
Web server can set the HTTPOnly flag on a cookie it creates, indicating the cookie should not be accessible on the client through JavaScript.

Logging for ASP.NET MVC 4: log4net

Logging is always an important part of a software project, tracing errors, passing audits, and allow monitoring etc.

I know how to use log4j in Spring framework, but today I have worked a little on C# MVC 4.

How to easily add logging into ASP.NET MVC 4 code base ?

I use log4net, and you can simply download it from VS NuGet store.

Steps for setting up after installation of log4net package:

1. Open  Global.asax.cs

protected void Application_Start() {
AreaRegistration.RegisterAllAreas();
FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
RouteConfig.RegisterRoutes(RouteTable.Routes);
BundleConfig.RegisterBundles(BundleTable.Bundles);
//add
log4net.Config.XmlConfigurator.Configure(new FileInfo(Server.MapPath("~/Web.config")));
 }

2. Open Startup.cs

[assembly: log4net.Config.XmlConfigurator(ConfigFile="Web.config", Watch = true)]

3. Open Web.config

<configuration>
<configSections>
//add
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>
//add
<log4net>
<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="app-log.txt" />
<appendToFile value="true"/>
<rollingStyle value="Size" />
<maxSizeRollBackups value="5" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date %logger (line:%line) - %message%newline" />
</layout>
</appender>
<root>
<level value="DEBUG"/>
<appender-ref ref="RollingFileAppender" />
</root>
</log4net>
</configuration>

4. Any namespace, say in Controller:

using log4net;
public class AccountController : Controller
{
//add
readonly ILog log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
public AccountController()
{
}
private bool try(){ log.Error("Error message"); }
...
}

The only thing missing here is that I have not tried to log Database SQL error. May update this later.

Mobile Web Development Notes from Google Udacity

1.Chrome Developer Tools:

DOM Inspector

CSS/JavaScript Live update

Network: HTTP requests and responses

Profiles: Record JavaScript object allocations to detect memory leaks

Mobile devices emulator

 2.Mobile UI and Viewport

Mobile First Design

Fixed width problem set viewport: width=device-width

Mobile app specific meta data

font-size: vw/vh

3.Fluid Design

Flexbox

Bootstrap CSS

Responsive Design:adapting to different devices use media queries

 4. Media Queries

@media  CSS3

https://developer.mozilla.org/en-US/docs/Web/Guide/CSS/Media_queries

example:

@media only screen and (min-device-width:768px) and (max-device-width:1024px)

@media only screen and (min-device-width:768px) and (max-device-width:1024px) and (orientation:landscape)

 5.Responsive Images

Image rendering in different screens: as beautiful as it is using minimal  bandwidth

Use SVG

Use Icon Font

Use Canvas

CSS gradient, borders, etc

Cheat: Use high quality but highly compressed image

 Server side to sniff client side to send correct definition image

Client side to request correct definition image

Use device-pixel-ratio

 CSS3

background-image: image-set();

http://css-tricks.com/which-responsive-images-solution-should-you-use/

 Helper tool Picturefill: https://github.com/scottjehl/picturefill

 7.Optimizing Web Performance

See Yahoo YSLOW.

 8.Touch event

Both mobile and desktop

Don’t :hover

Don’t disable mouse support

Do use large hit target

Don’t detect ontouchstart

Fix click delay, i.e. quick twice click for zoom: -user-scalable=no, but can’t scale view

Support touch directly

Two touches spontaneously: add id

IE: Pointer Events

http://msdn.microsoft.com/en-us/library/ie/dn433244%28v=vs.85%29.aspx

9.Input

HTML5:

color, date, datetime, datetime-local,email,month,number,range,search,tel,time,url,week

This will trigger different keyboard in mobile device.

pattern = “regex” for simple client side validation, all input in form must validate in server side.

10.Device Access

navigator.getUserMedia

Sensors in mobile

Camera

Audio

GPS/Geolocation

Compass

Orientation

Motion

 11.Offline and Storage

navigator.online detection is not stable

Client cache

Server cache

AppCache vs HTTP cache

Cache manifest

sessionStorage

localStorage

IndexedDB

http://www.html5rocks.com/en/tutorials/appcache/beginner/