This blog post is the first one in a series of posts covering how to send free SMS messages using the Ung1881 web site. This first post discuss how to sign into a "Forms Authentication" protected ASP.NET web site, store an authentication cookie, and programmatically post web forms. Even if you're not interested in the free SMS messages you might still find the article interesting. The next part will be on how to use the .NET Class Library in a Windows Vista Gadget.

Unlimited free SMS messages, exposed as a .NET Class Library… Sounds to good to be true? Well, It's not, but as with everything else that's free (except Open Source?) it has a catch.  You can only send messages containing 130 characters. The last 30 characters are reserved for commercials. The good thing is that you can send as many messages as you want, and your own number show up as the sender. The service is offered by "Opplysningen 1881"  (Norwegian Number Enquiry) through their new portal "Ung1881", a page aimed at young people. In order to use their SMS service you need to be a have a Norwegian social security number, and a phone number. They need your social to do a lookup and verify that you're a real person. Even though you might not be able to register for an account at "Ung1881" you might still find the code and article useful since the concepts apply for any web application.

I got interested in the "Ung1881" after Håvard pointed me to it some weeks back. After doing some re-search it turned out someone had written a Mac OSX Widget letting you send SMS messages directly from your Mac desktop. My immediate idea was to port this widget to the Windows Vista Sidebar, but after looking at the JavaScript code of the widget I soon realized this was a bad idea. The Mac OSX Widget is depending on a third party server doing the screen scrapping server side, so all your SMS messages (including your username and password) are sent through a third party using HTTP GET like this: http://www.theretard.net/smss.pl?u=username&p=password&n=number&m=message. I don't know the guys behind theretard.net, but there's no way I'm trusting them with my username, password and every SMS message I ever send. Just imagine the password harvesting possibilities! On top of this they don't even use HTTPS for communication.

I decided I had to write a .NET Assembly wrapping the "Ung1881" service, so that I could re-use the functionality in other applications, like a Windows Vista Sidebar Gadget, a console application, Outlook 2007, PowerShell etc.

The technique of accessing web sites programmatically is commonly refereed to as  screen scrapping. Since the Ung1881 portal doesn't offer any kind of programming interface, you need to tap in to it at the HTTP level, and "pretend" that you're a regular user accessing the page with a browser. Since you're depending on the actual HTML structure page you're application might become unstable, and if the owner of the source you're scrapping change their HTML it might break your application. You also need to be careful that you're not violating copyright regulations, by for instance downloading weather information and displaying it as your own. In the case of "Ung1881" terms don't say anything about "automatic access", and as long as you have a valid account they still get their 30 character commercials included in every SMS message you send. I don't think you'll get into trouble by using this code, but if you do don't blame me!

The "Ung1881" portal is running on the .NET-based Content Management System EPiServer, and uses ASP.NET Forms Authentication to authenticate users on the portal. Forms Authentication normally works by using an authentication cookie. When you logon the server authenticates your username and password. If your credentials are valid, it issues a cookie your browser sends with subsequent requests to the site. So in order to wrap the SMS service in a C# class you basically need to write a "non visual" Web Browser. You need to make a request to their login page, use HTTP POST to post a username and password to the server and store the authentication cookie sent back in the response. On subsequent requests you need to attach the authentication cookie in order to access protected pages (like the send SMS page).

The first thing I did when implementing the "Ung1881" proxy class was to analyze the HTTP traffic going between server and client. There are several tools you can use to monitor network traffic at different levels. My preferred HTTP monitor is Fiddler. Fiddler act as a HTTP Proxy, so all you're HTTP requests are routed through Fiddler. Using Fiddler you can look at the raw requests, the headers, the posted form values etc. New in Fiddler v2.0 is support for HTTPS, which is excellent since the "Ung1881" portal is using HTTPS for secure communication. Another excellent tool for network monitoring is oSpy, written by Ole André Vadla Ravnås - a brilliant programmer, reverse engineer and in general a really nice guy!

The above screenshot shows all the HTTP traffic between Internet Explorer and the "Ung1881" web server. As you can see in the left column the site is quite "chatty" because of all the pictures, advertisements, script, style sheets etc. If you enter your username, password and hit the login button, and look at the request made to the URL https://www.ung1881.no/default____3.aspx you can figure out which form elements that are being posted to the server. The site contains quite a few form elements, but the interesting parts are the following:

Content-Disposition: form-data; name="defaultframework:login:tbxUsername"
myusername

Content-Disposition: form-data; name="defaultframework:login:tbxPassword"
mypassword

Content-Disposition: form-data; name="defaultframework:login:LoginButton.x"
0

Content-Disposition: form-data; name="defaultframework:login:LoginButton.y"
0

Content-Disposition: form-data; name="__VIEWSTATE"
dDw5NDE0MjMxMzc7O2w8ZGVmYXVsdGZyYW1ld29yazpsb2dpbjpMb2dpbk
J1dHRvbjtkZWZhdWx0ZnJhbWV3b3JrOnNlbmRzbXM6YnRuU2VuZDs+Pp5c
kvQYH64y3L+4/9ebcQzi7GmX

I've highlighted the names of the form elements. You could have figured out most of this just by looking at the HTML source, but I find it easier to use Fiddler. The thing that might not be too obvious just by looking at the HTML source, is what form value get posted when you click the login image button. The HTML fragment looks like this:

<input type="image" name="defaultframework:login:LoginButton" id="defaultframework_login_LoginButton" class="..." src="..." alt="" />

If you look at the HTTP POST values above you can see that there are two values posted back to the server when you click the login button, defaultframework:login:LoginButton.x and defaultframework:login:LoginButton.y. Both have the value 0. I took a few seconds before I figured what was happening, but apparently the HTML specifications says that <input type="image"> should post back the x and y values with the coordinates of where the user clicked the image. For accessibility reasons this is deprecated, and if you need x and y values from an image you should use an <imgemap> element instead. In order to confirm with the HTML standard the browser posts an x and y value back to the server.

Another important thing to note is that "Ung1881" is an ASP.NET application, and therefore is depending on the "__VIEWSTATE" form element. This hidden form value holds an encoded string representing the state of the application on the server. If we don't include this value in our scrapper the ASP.NET application can't figure out what state it's in, and how to process the request correctly.

If you look at the response sent back from the server you see that the server issues three cookies:

ASP.NET_SessionId=jnq5hh45zuuu0xyigc1jtm45; path=/

.EPiServerLogin=6E1DBAC98D4073F956DA2000EBF122056E335441CCC7756F98B2
A229387EC47E446701A29D36C4497553A5409F40322142C84B140E93C25CFC468C81
5B25CCC35648CE9B03C96XXXXXXXXXXXX; path=/;HttpOnly

ung1881.no=3568924299.20480.0000; expires=Sun, 25-Mar-2007 16:22:15 GMT; path=/

The interesting part is the .EPiServerLogin-cookie which is the authentication cookie. By adding this cookie to my subsequent requests, the server can tell that you're an authenticated user.

Now that we know what's going on when login into the page it's time to send an SMS message. I click the "Send SMS" link on the page, which takes me to the following URL: https://www.ung1881.no/Templates/SMS____24.aspx. I enter a phone number and message, and hit the button to send the message. The requests gets picked up by Fiddler, and by analyzing the request I figure out which form elements I need to post to the server in order to send a message:

Content-Disposition: form-data; name="defaultframework:_ctl2:Smssend:txtPhonenumber"
myNumber

Content-Disposition: form-data; name="defaultframework:_ctl2:Smssend:txtText"
myMessage

Content-Disposition: form-data; name="defaultframework:_ctl2:Smssend:butSend.x"
46

Content-Disposition: form-data; name="defaultframework:_ctl2:Smssend:butSend.y"
9

Content-Disposition: form-data; name="__VIEWSTATE"
*LARGE CHUNK OF STATE*

So after analyzing the application I now know the following about the communication between browser and server:

• The login URL is: https://www.ung1881.no/default____3.aspx
• The username form element is named: defaultframework:login:tbxUsername
• The password form element is named: defaultframework:login:tbxPassword
• The login button posts two values: defaultframework:login:LoginButton.x and defaultframework:login:LoginButton.y
• The server issues an authentication cookie named .EPiServerLogin
  
• The send SMS URL is: https://www.ung1881.no/Templates/SMS____24.aspx
• The number form element is named: defaultframework:_ctl2:Smssend:txtPhonenumber
• The message form element is named: defaultframework:_ctl2:Smssend:txtText
• The send SMS button posts two values: defaultframework:_ctl2:Smssend:butSend.x and defaultframework:_ctl2:Smssend:butSend.y

Now that I know everything I need to know about the form elements It's time to write a C# class wrapping the site. The code is fairly well commented, so I won't describe it in detail. The key classes involved in the scrapper are the HttpWebRequest, HttpWebResponse, CookieContainer, StreamReader and StreamWriter classes. Another interesting part of the code is the private GetViewState method which extracts the view state of the page so that you can post it back to the server.

To test the class I've created a simple console application accepting a username, password, number and message as it's argument. Using the console application you can send SMS messages by typing the command: "SMS username password number message". If you add the folder containing the .EXE to your environment path you can send SMS messages from any command prompt.

So that's about it. A simple C# library allowing you to send unlimited free SMS messages through "Ung1881". You can choose to either download the compiled binaries or the source code. In the next part of this article/post I'll discuss how to use the library in a Windows Vista Sidebar Gadget.

If you have any questions, comments or bugs, or find the library useful, please drop me a comment!