Sunday, April 21, 2013

Wake on Lan in C# and Windows 8

About 8 years ago I was writing scripts to run on a network with over 130.000 computers (of which 5000 I administered).
The scripts ran 24/7, parsing computer's inventory log files, which they sent to a central server. It was  possible to detect and fix a whole bunch of issues, most of the time even before a user would notice something was wrong.

Note that most of those computers were running Windows NT 4, including the domain controllers. The task to install application in all those computers and keep their anti-virus signature up-to-date was not as trivial as it is today. There were times we needed to perform tasks on computers that weren't even switched on. And I must admit, back then I was quite proud of the solution I came up with for this particular case. Although it's not my goal to go into details on how I managed to get any of those 5000 computers, spread in 130 different offices, powered-on at any time; I want to write a little about the core of the solution: Wake on Lan

Whether to be able to power on your computer at home when you are away, or to manage a corporate network, the ability to switch a computer on out of sending a magic packet is at least quite interesting. It does require basic knowledge of computer networks and hardware to understand it, but to make it work, all you need is to know how to find your computer's MAC address. That's all you need to use WoL on a local network. However if you are willing to use it over the Internet, you'll need the IP address of the gateway of which the target computer belongs to, and have that gateway configured to forward the magic packet to the private network. The Port number is used in this case (if you use NAT, you'll need it), so that you can forward incoming UDP datagram on that predefined port of your router to an internal network IP address, or even the broadcast address if you'd like.

If the computer is switched off, it's likely the network switch's CAM table won't have an entry to your target computer's MAC address, and once your router forwards the datagram received from the external interface to your internal network, to the specific IP address you defined on the forwarding rule, the network switch will broadcast that to all ports.

You'll find many over architectured implementations of Wake on Lan out there, even in C#. But the fact is that it's really simple thing.

Let's see some C# code:


using System;
using System.Net;
using System.Net.NetworkInformation;
using System.Net.Sockets;
using System.Text.RegularExpressions;

namespace BrunoGarcia.Net
{
    public sealed class WolManager
    {
        const int _payloadSize = 102;

        public static void SendMagicPacket(string mac, IPEndPoint ipEndPoint)
        {
            var macBytes = PhysicalAddress.Parse(mac).GetAddressBytes();

            var payload = new byte[_payloadSize];
            Buffer.BlockCopy(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF }, 0, payload, 0, 6);
            
            for (int i = 1; i < 17; i++)
                Buffer.BlockCopy(macBytes, 0, payload, 6 * i, 6);

            using (var udp = new UdpClient())
                udp.Send(payload, _payloadSize, ipEndPoint);
        }
    }
}

As you can see, it's a fire-and-forget operation, a single UDP datagram. The MAC address of the target machine goes in the payload. As I mentioned before, the port is not really important if you are sending the packet on the same subnet of your target computer (without the need of routing).  Port on wikipedia you find to be 7 or 9. However 0 can be used:


WolManager.SendMagicPacket("B8-AC-6F-59-56-55", 
    new IPEndPoint(IPAddress.Parse("255.255.255.255"), 0));

This is fun, and I've used it to power on my computer at home from the Internet a few times. But now, when I got Windows 8, the fun wasn't quite working.

This KB from Microsoft describes the change:

Windows 7:  In Windows 7, the default shutdown operation puts the system into classic shutdown (S5) and all devices are put into the lowest power state D3. Wake-On-LAN is not officially supported from S5 in Windows 7. However, some network adapters can be left armed for wake if enough residual power is available. As a result, wake from the S5 state is possible on some systems where enough residual power was supplied to the network adapter even though the system is in S5 and devices are in D3. 

Windows 8:  In Windows 8, the default shutdown behavior puts the system into hybrid shutdown (S4) and all devices are put into D3. Remote Wake-On-LAN from hybrid shutdown (S4) or classic shutdown (S5) is unsupported. Network adapters are explicitly not armed for Wake-On-LAN in both the classic shutdown (S5) and hybrid shutdown (S4) cases because users expect zero power consumption and battery drain in the shutdown state. This behavior removes the possibility of spurious wakes when explicit shutdown was requested. As a result, Wake-On-LAN is only supported from sleep (S3) or hibernate (S4) in Windows 8.

What a bummer. Anyway, not time to give up, right? Digging a bit deeper one understands better the changes, and enabling wake on lan on windows 8 is possible again:

Check out this post from Phil Pendlebury which talks about it.

Wednesday, March 13, 2013

Azure blog, events in Europe

It's not a secret I'm a Microsoft fan. I work with their development platform and I like it.
I also read books from Microsoft Press as I have mentioned before, and have lots of "blogs.msdn" on my RSS feed.

From the Azure blog, I'm always checking events (although they never come to Prague).
Recently while browsing the list of events in Europe I notice one very close to Prague: Mumbai, India


Perhaps I can attend this one. Just a short train ride. :)

Monday, March 4, 2013

Moq Return method not available after Setup

If you are familiar with the mocking framework Moq, you're used to call Setup with the overload taking a Func<T, TResult> and expect after that the Return<TResult> method to be available. And it's normally there.

However, I just ran into an interesting scenario, where calling the correct overload did not make available the Return<TResult> method.

In my case the code being mocked is a dependency that makes a request on a webservice.
A mockup of the wrapper class is created, and the Load method, which returns an XDocument, is setup.

But Return wasn't available:



Interesting that the overload resolution resolved my Func<T, TResult> to Action<T>, therefore ISetup<T> was being returned instead of ISetup<T, TResult> even though the method I setup had return type!

I inspect the method I setup to double check the return type defined:


Yes, it's missing a reference to System.Xml.Linq.
The Unit Test Project template doesn't include a reference to this assembly (and that makes sense).
Well, I added it, it works now.


Saturday, February 23, 2013

SignalR: Using a Hub instance not created by the HubPipeline is unsupported

When you need to push data to a SignalR hub from outside the hub (from a Controller for example), don't try to create a new instance of the Hub, like I did. Otherwise you'll see this nice exception:

"Using a Hub instance not created by the HubPipeline is unsupported"

From Microsoft.AspNet.SignalR.Core

Instead, the hub context must be retrieved:

var context = GlobalHost.ConnectionManager.GetHubContext<HubType>();
context.Clients.All.Whatever();

Saturday, February 16, 2013

Regex to match executing a stored procedure within a stored procedure

Unless you're working with a custom string format, which requires you writing a regular expression, I suppose (suggest?) you'll look it up online, right?

If you need a Regex to match an IP, MAC or an E-mail, would you spend time writing it? Chances are that you might leave room for false positives and/or false negatives, unless you really test it. That's why it's common to look it up online.

I needed to take a list of proc names and parse thousands of create procedure scripts, looking up if anything from the input list was used (executed) from those procs. A procedure executing another procedure.

This kind of call can be made in several different ways. It might have or not the database name (cross db call), server name (linked server), it might have or not the EXEC/EXECUTE keyword. It might have or not brackets or set the result to a variable.

Since I looked it up and couldn't find it, I wrote it.
In case you need it, here you go:

^\s*((exec(ute)?)\s+)?(@\w+\s+=\s+)?((\[?\w+\]?\.{1,2}){1,3})?\[?p_storedProcedure\]?\s.*$

Note that if you use SQL Server with default collation, proc names are case insensitive. So make sure you let your Regex engine know it should ignore cases! Otherwise you'll have to mind the case of the EXEC/EXECUTE keywords anyway.

Also note that cross database and linked server calls will also match. Some examples of valid proc calls (that will match against the Regex) are:

p_storedProcedure -- comments
p_storedProcedure @id, @anotherParam
[p_storedProcedure] @id, @anotherParam
EXEC p_storedProcedure @id, @anotherParam
EXECUTE p_storedProcedure @id, @anotherParam
EXEC p_storedProcedure
EXEC p_storedProcedure -- comments
EXEC [p_storedProcedure] @id, @anotherParam
EXEC dbo.p_storedProcedure @id, @anotherParam
EXEC dbo.[p_storedProcedure] @id, @anotherParam
EXEC anySchema.[p_storedProcedure] @id, @anotherParam
EXECUTE dbo.[p_storedProcedure] @id, @anotherParam
EXEC [dbo].[p_storedProcedure] @id, @anotherParam
EXEC [dbo].p_storedProcedure @id, @anotherParam
EXEC DBTEST..p_storedProcedure @id, @anotherParam
EXEC DBTEST..[p_storedProcedure]
EXEC DBTEST.dbo.p_storedProcedure @id, @anotherParam
EXEC [DBTEST]..[p_storedProcedure] @id, @anotherParam
EXEC [DBTEST]..p_storedProcedure @id, @anotherParam
EXEC [DBTEST].[dbo].[p_storedProcedure] @id, @anotherParam
EXEC [LINKEDDATABASE].[DBTEST].[dbo].[p_storedProcedure] @id, @anotherParam
EXEC LINKEDDATABASE.DBTEST.dbo.p_storedProcedure @id, @anotherParam
EXEC LINKEDDATABASE.DBTEST..p_storedProcedure @id, @anotherParam
EXEC @paramName = [dbo].[p_storedProcedure] @id, @anotherParam
EXEC @paramName = dbo.p_storedProcedure
EXEC @paramName = p_storedProcedure @id, @anotherParam
EXECUTE @paramName = p_storedProcedure @id, @anotherParam
EXECUTE    @paramName     =     p_storedProcedure     @id, @anotherParam

Wednesday, December 12, 2012

Pitfalls on WIF+SAML2 and Selenium

WIF and SAML 2.0


First some background: There is a known issue on WIF (Windows Identity Foundation) for SAML 2.0 that generates cookies with a name being a GUID and the value, base64 encoded data that grows every SAMLRequest the module handles. The decoded value looks like: 0;1;2;3;4;5;6;7;8;9;10;11;12;13;14;15
It starts with small ones but get really, really large.



Every client gets one of these cookies and each time they are bigger, to the point that when they are sent back to the server, an HTTP error is thrown: HTTP 400 - Bad Request (Request Header too long)

This msdn link has a comment with the first steps to take in case you end up with this problem. They are very straight forward and we did them even before ending up on that msdn page. Regarding their forth step (final "fix"), in our case, it was decided a different solution.

The solution here was to remove the cookies before they would be sent out to the user in the first place. This way, even though for some really short time the cookie existed in memory at the server, the client never got to know of its existence. Do achieve that, login and logout gotta be changed. That is: SignIn and RedirectingToIdentityProvider events from the Saml2AuthenticationModule. At that point in the event pipeline, the underlying Microsoft WIF code had already added the cookies to the Response, which gives us opportunity to remove them before the headers are sent out to the client.

Which takes us to Selenium:


The final solution had to be tested before dropping new build to production. And to test it, we had to reproduce it. The issue was not known during Dev or QA phases/environments, it did not happen, so the first step was to be able to reproduce it on a controlled environment.

Basically the idea was to use Selenium to simulate few dozens of users logging in and off in parallel until a cookie matching a GUID (plus a number?) would be received by one of the clients. There was no need to let it grow to the point of having: HTTP 400 - Bad Request (Request Header too long)

For that, I wrote a small application to spawn a thread for each IWebDriver (threads from the pool were conflicting the Drivers), each logging in with a different user account, removing the cookies (so user would be challenged again) and starting over.

The code would detect the existence of the cookie and stop the test, but to make visible (the cookies in and out) we can load Selenium driver with Firebug enabled and the cookie tab enabled and visible as default.

That goes like:

const string firebug = @"firebug-1.10.6-fx.xpi";
IWebDriver driver;
if (includeFireBug && File.Exists(firebug))
{
 var profile = new FirefoxProfile();
 profile.AddExtension(firebug);
 // Set default Firebug preferences
 profile.SetPreference("extensions.firebug.currentVersion", "1.10.6");
 profile.SetPreference("extensions.firebug.allPagesActivation", "on");
 profile.SetPreference("extensions.firebug.defaultPanelName", "cookies");
 profile.SetPreference("extensions.firebug.net.enableSites", true);
 profile.SetPreference("extensions.firebug.cookies.enableSites", true);
 driver = new FirefoxDriver(profile);

I mentioned the code would check the cookies to look for the GUID one, and with selenium API, it's very simple to do so:

Guid test;
if (driver.Manage().Cookies.AllCookies.Any(p => p.Name.Length >= 37 
    && Guid.TryParse(p.Name.Substring(0, 36), out test)))
{
...

Just checked if it's big enough to be GUID, then tried to parse the GUID part of it (note it appends some number to sequentially divide them into 2k sized each).

Two domains involved in this test. The service provider, let's call: service.com
And the identity provider: idp.com

Initially I set the IWebDriver Url property to the service provider: service.com
Find the element for Login and fire a click. That would call the SAML module that would redirect the client to the identity provider: idp.com

The login and password input elements would be filled up and login button triggered in the IdP page.
At this point, the session cookie from the IdP was sent to the browser, under idp.com domain, and client redirected back to service.com. SAML flow finished and session cookies from service.com also sent to the client.

That's all we need to reproduce the issue. However, these steps had to be done over and over, several times until the issue would happen. Particularly in our case, Logout was not possible since the accounts used were test account and thus not validated, so simply deleting the cookies would enable us restart the flow (and save us some requests/time). But this means deleting cookies from both service.com and idp.com.

Using Selenium API, I wrote:

driver.Manage().Cookies.DeleteAllCookies();

Even though the method is called DeleteAllCookies, it deletes all cookies from the current domain on which the WebDriver is located. In this case, service.com, since user just landed after the SAML login.
Looping the Cookies collection from within the WebDriver obviously would return only the cookies from the current domain.

It was the time for a second maneuver:
Setting the Url property of the WebDriver to anywhere under the idp.com domain that wouldn't return with a redirection, and call again DeleteAllCookies. That simple.
I browsed the root of the domain, without any resource id, which returned 403.14 - Directory listing denied. That was enough to run a code like:

// right after login flow finished (landed on service.com, logged in)
driver.Manage().Cookies.DeleteAllCookies(); // deletes service.com cookies
driver.Url = "idp.com";
driver.Manage().Cookies.DeleteAllCookies(); // deletes idp.com cookies

After that the flow could be re-initiated. After few hundreds of times, we could reproduce the issue, add the fix, run the test again with thousands of logins, without any issues. 

Saturday, November 10, 2012

Top level domains and punycode with C#

Punycode is used to encode Unicode characters into ASCII for IDN (Internationalized domain name).

On the RFC 3492 you'll find:

"Punycode is a simple and efficient transfer encoding syntax designed for use with Internationalized Domain Names in Applications (IDNA). It uniquely and reversibly transforms a Unicode string into an ASCII string."

Now if you are looking for validating TLD (Top level domains), you must have that information in mind. The ICANN list of TLD also contains the IDN ccTLD that started to be included in 2010.

Some Punycode encoded examples from that list:
XN--0ZWM56D
XN--11B5BS3A9AJ6G
XN--3E0B707E
XN--45BRJ9C
XN--80AKHBYKNJ4F
XN--80AO21A
XN--90A3AC
XN--9T4B11YI5A
XN--CLCHC0EA0B2G2A9GCD
XN--DEBA0AD
XN--FIQS8S
XN--FIQZ9S
The prefix is XN-- makes it easier to identify the Punycode enconded strings.

Luckily the since version 2.0, the .Net Framework offers a class to deal with IDN (Punycode and the Nameprep it has to do prior to encoding):

System.Globalization.IdnMapping.

My goal was to receive a TLD (string) and validate it against the ICANN list of TLD. My first snippet throw an exception on line 4:
var tld = ".ਭਾਰਤ";
if (Regex.IsMatch(tld, @"[^\u0000-\u007F]"))
{
    tld = _IdnMapping.GetAscii(tld);
}
Exception message was: IDN labels must be between 1 and 63 characters long.

My speed reading techniques are quite bad.. In fact I don't have any. Sometimes I just focus on what I believe to be the most important part of the message (in this case "1 and 63 chars long" which didn't make sense) and I ended up missing something important (IDN labels).

I googled the exception, finding only these very useful (?!?) Exception translation websites and nothing more.
Only to better read the message and realize that the catch was that IdnMapping works with domain name labels:

"A constituent part of a domain name. The labels of domain names are connected by dots. For example, "www.iana.org" contains three labels — "www", "iana" and "org". For internationalized domain names, the labels may be referred to as A-labels and U-labels."

Therefore, my input was simply broken considering it started with a dot. If you are looking to validate the complete list of TLD, including ccTLD, or even the complete domain with multiple labels supporting IDN, the IdnMapping class is a go. However, make sure your code does not have leading or trailing dots by having it Trim('.') or something.


Regarding the IDNA versions, on the .Net framework prior to version 4.5 works with version 2003. Now if you are running .Net Framework 4.5 on Windows 8, the IDNA 2008 will be used.