Back to work

I was on vacation for nearly a month, sadly unpaid.

I have joined NUS to study master courses of Information Systems in computing since last week.
It has a concentration on IS management, and requires lots of reading of researches and presentation.

However, I have determined to continue my self-study on web development knowledge, e.g., JavaScript, CSS, etc.

  1. Working
  2. Research on school projects
  3. Reading ES6 in Depth
  4. Re-read Data Structures and Algorithms following the CareerCup book
  5. Continue CodeEval challenges

A new journey begins…

OWASP top 10 vulnerabilities 2013

When doing web app security analysis/audit, people often refer to OWASP, the Open Web Security Project.

Below is a short summary of top 10 vulnerabilities for web app in 2013.

A1 – Injection: SQL/NoSQL injection, OS commands, Xpath, etc
A2 – Broken Authentication and Session Management: authentication design flaws
A3 – Cross-Site Scripting (XSS): data sent to server not validated/escaped
A4 – Insecure Direct Object References: authorization design flaws
A5 – Security Misconfiguration: system level configuration flaws, like exposure of internal web services
A6 – Sensitive Data Exposure: lack or weak encryption of important data like password
A7 – Missing Function Level Access Control: like URL route not secured, easy to guess different passing of parameters
A8 – Cross-Site Request Forgery (CSRF): Attacker creates forged HTTP requests and tricks a victim into submitting them via image tags or XSS
A9 – Using Known Vulnerable Components: using third-party libraries can be dangerous
A10 – Unvalidated Redirects and Forwards: unchecked redirects

IBM has a site to demo for the details, and marketing of their product, IBM Security AppScan.

A bug in ArcGIS REST Service

We run ArcGIS servers in two different environments DEVELOPMENT and UAT , and the guy who set up the infrastructure has gone.
Last week we deployed the working app into the UAT, and it didn’t work as supposed as in DEVELOPMENT.

Specifically,  we had problem with the ArcGIS login process. We use ArcGIS REST API token generation for login authentication, and the app host address as HTTP referer.
Details see http://resources.arcgis.com/en/help/arcgis-rest-api/index.html#/Generate_Token/02r3000000m5000000/.
After user logs in, we use the generate Sitemap in REST service to get a list of access-controlled layers for the user.
We don’t use the REST method as it only returns the current folders and then again requires ajax under the folder to get all the layers, it slows down the whole app.

In DEV, we successfully get the sitemap XML with all the layers as shown like this.

However, we could not manage it in the UAT, and it always returned an empty XML, i.e. no layer services at all.

Token returned from UAT server was correct, and there was nothing wrong with code and configuration. We debug the login module, the token generation process and any possible out-dated codes. Nothing worked.

The bug cost me one week to catch.  See Server 1 as DEV and Server 2 as UAT, something is missing:

agsbug

I doubt that the guy who implemented the REST interface was different from the one who implemented the Sitemap interface. The later just ignored the whole folders when there was None Services, but this None only means there is no service under the root. Damn!

A few thoughts

1. I went to a short and unexpected interview yesterday.

There was a quiz, maybe abstracted from somewhere  in the web of HTML and JavaScript.

What’s the difference between conforming and non-confirming HTML ? I don’t know. 

Why meta charset must within 1024 bytes of a HTML doc ? I never encounter this error before, and so I don’t know.

I think I screwed it up, and I don’t blame myself or anyone else.

Without  memorizing or encountering some practical problems in software development, you could never fully answer those kinds of problems related to W3C “DEFINITION”.

They call it fundamentals. However, you can’t really solve real-work problems just by knowing the definition.

It’s better to fully read through all the W3C definitions.

So Just Read.

2. I am 26.

Working as a contracting staff with a not-so-low salary.  BUT I want to achieve more.

So I got myself enrolled to NUS Master of Computing in Aug. Part-time studying for two years with fees of 13K SGD.

Welcome, Exams!

How to calculate the ROI? Can it be better than my ROI in stocks > 10% annually ?

I should find a girlfriend.

3. Too much work or Too many managers

Wherever I go, there are people who are really busy, and people who look very busy.

Busy people are actually doing the job, and looking-busy guys are managing them.

Can or Can not ? Asked the manager.

4. Chinese A shares BULL

If you have invested 10K, then got doubled in less than a year. Seems really good.

Hold.

BUT be careful, because all share-prices are doubled.

Pure luck can win at some time, but will lose in the long run.

NOBODY CARES!

Chasing the pavements.

Bypass IdentityManager Popup in ESRI ArcGIS JavaScript API

When using ArcGIS secure layers in JavaScript API 2.5+, the default IdentityManager will come out as a login process for web app users.Please see example at http://developers.arcgis.com/javascript/samples/widget_identitymanager/ What if I already have a login for a web app, and do I need to login again for the map layers? What can be done to avoid the IdentityManager Popup ? Login use ArcGIS REST API: see http://resources.arcgis.com/en/help/arcgis-rest-api/index.html#//02r3000000m5000000 And then bypass the IdentityManager Popup.

//just use AJAX, you can put this login in any page
var timeout = 1440, referer = hostname + '/arcgis/rest';
var postData = {
    'username': username,
    'password': password,
    //'client': 'requestip', //can't work if using this type of request
    'referer': referer,//should be the web app host name
    'expiration': timeout,
    'f': 'json'//,
    //'encrypted': 'true' //if need to enable, please add the necessary js files, see generateToken page source code from ArcGIS
};
//processing data encryption
//postData = encryptFormData(postData, modulus);
$.ajax({
   url: url,//hostname/arcgis/tokens/generateToken
   cache: false,
   type: 'POST',
   dataType: 'jsonp',
   data: postData,
   crossDomain: true,
   success: function (response) {
    //console.log(response);
    if (response) {
        if (response.error) { // access error
         return false;
        } else {
         //store username and token in cookies
         setCookieName("arcgisusername", username, 12);
         setCookieName("arcgistoken", response.token, 12);
        }
     } else {
        return false;
      }
   },
   error: function (error) { // system error
    return false; 
   }
});

//bypass the popup
//require IdentityManager
//now build a identitymanager json object to init the object
var now = +(new Date());
var expires = now + (timeout*60000);
var imObject = {
"serverInfos": [
{
"server": hostname,
"tokenServiceUrl": hostname + "/arcgis/tokens/",
"adminTokenServiceUrl": hostname + "/arcgis/admin/generateToken",
"shortLivedTokenValidity": timeout,
"currentVersion": 10.22,//update necessary
"hasServer": true
}
],
"oAuthInfos": [],
"credentials": [
{
"userId": getCookieName('arcgisusername'),
"server": hostname,
"token": getCookieName('arcgistoken'),
"expires": expires,
"validity": timeout,
"ssl": false,
"creationTime": now,
"scope": "server",
"resources": [
hostname + '/arcgis/rest/services'
]
}
]
};
identityManager.initialize(imObject);

//Please remember that if you need to query some layer data using ajax or using some layer URL,
// you need to manually append the '&token=sometokenfromserver' for the URL

I found another hack to bypass the ArcGIS IdentityManager popup or a new method to access secure layers.

This requires coding in .NET/C#. I figure out that ArcGIS server use request cookie name agstoken as an authentication method for accessing secure layers.So if I can create a login form and post to the Login.aspx and send the response with cookie contains agstoken which I can freely request from ArcGIS services using the REST API.


using System;
using System.Collections.Generic;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Net;
using System.IO;
using System.Text;
namespace Login
{
    public partial class _Login : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {   
            //code for normal user login, if failed, then redirect to login page
  	    string h = "hostname";
	    //h = Request.UserHostAddress;//may not work as direct ip address
            Response.AddHeader("Access-Control-Allow-Origin","*"); //may tighten the access
            Response.AddHeader("Access-Control-Allow-Methods", "POST");
            Response.AddHeader("Access-Control-Allow-Headers", "Content-Type, Accept");
	    //cookies
	    HttpCookie c2 = Request.Cookies.Get("agstoken");
	    HttpCookie c3 = Request.Cookies.Get("agsapperror");	
	    string timeout = "1440";//mins
	    string u = Request["u"].ToString();
	    string p = Request["p"].ToString();
	    string referer = h;
	    string tokenURL = h + "/arcgis/tokens/generateToken";
            string type = "application/x-www-form-urlencoded";
            string token = "";
            WebClient wc1 = new WebClient();	
	        try{
	          string loginData = "username="+u+ "&password="+p+     
                       "&f=json&expiration="+timeout+"&encrypted=false&referer="+referer;          
                  wc1.Headers[HttpRequestHeader.ContentType] = type;
	          string loginContent = wc1.UploadString(tokenURL, loginData );
	          //check result      
                  if(loginContent.Contains("error")){
			if(c3 == null){
			     c3 = new HttpCookie("agsapperror");
			     c3.Value  = "userpasswordwrong";
			     c3.Expires = DateTime.Now.AddMinutes(10d);
			     Response.Cookies.Add(c3);
			}
			   Response.Redirect("/");
		               
		   }else{ 
		          //push user token for accessing secure layers
			  token = getUserToken(loginContent);
                          if(c2 == null){
				c2 = new HttpCookie("agstoken");
				c2.Value  = token;
				c2.Expires = DateTime.Now.AddMinutes(1440d);
				Response.Cookies.Add(c2);
			  }
               	         Response.Redirect("/index.html"); 
		   }								   
		   
	       } catch (WebException e1) {
                   //Response.Write("{\"error\":\"network connection error\"}");
		   Response.Redirect("/");
            }
        }
      
	public string getUserToken(string tokens){
	    string token = ""; 
	   if(tokens.Contains("token")){
               string [] r = tokens.Split(new string[]{"\",\""},StringSplitOptions.None);
	       if(r.Length > 0){
                   string part1 = r[0];
                   string [] r2 = part1.Split(new string[]{"\":\""},StringSplitOptions.None);
                   token = r2[1];   
               }
           }
	  return token;
	}
    }
}

GIT Workflow Basics

  1. Simplest Centralized Workflow
     Not preferred

    1. One origin master, all members clone from the remote master
    2. All members work on the local master
    3. All members need to first pull the remote master, and then rebase to push to the master
    4. Members manage conflicts in the 3rd step case by case
  2. Feature Branch Workflow

    1. One origin master, all members clone for the remote master
    2. All members create a new branch to work on locally, if necessary, push only one branch for each member to the remote server
    3. All members must merge the new branch with master to push
    4. Members manage conflicts in the 3rd step case by case