Friday, January 31, 2014

How to implement re-fulfillment for products with large catalog of offers in Windows Store App

This is what is said in an article titled “How to manage a large catalog of in-app offers (Windows Store apps using C#/VB/C++ and XAML)” in “How-to” session of “Dev Center- Window Store Apps”

As mentioned earlier, the Windows Store only uses provided offer info to populate the PurchaseResults, and does not create a persistent association between a large catalog offer and Windows Store product listing. As a result you need to track user entitlement for large catalog offers, and provide offer-specific context (such as the name of the item being purchased or details about it) to the user outside of the RequestProductPurchaseAsync operation.
You can get this article at

With that, is it correct to conclude that the store’s in-app purchase framework does not keep track which offer was purchased within a product by which user?
"You" as an app developer, need to track what offer was purchase. If "you" want to offer re-fulfillment after user uninstall/re-installed or install the app in other device they own. The "tracking" needs to be done outside of the app. In another word, the information needs to be stored in a service that is accessible to the app.
In order for "you" to keep track who purchase what offer of which product, "You" need to know who is the current user.
But UserInformation class only gives "you"  DisplayName, LastName and FirstName, through GetDisplayNameAsync, GetFirstNameAsync and GetLastNameAsync methods.
This is good from privacy point of view, as these pieces of information cannot be used to identify the user. So privacy is protected. However, the question here is how do "you" as the developer to track what offer was purchased by who?
I think this is a gap in in-store purchase framework.
I am putting this question here and hope someone can prove me wrong...

If what I described above is accurate. I would bold enough to offer the following as suggestion:

A reasonable solution would be Microsoft modify its in-app purchase framework to provide one more method on licenseInformation class, say IsPurchased or IsActive and it has two overloadings. One takes only one string which is ProductId, the other one takes 2 parameters, ProductId and OfferId).

Response from Microsoft Technical Support:

Sad news is that in such cases, you will have to keep a track of offers being purchased by the users. Ideally, a web service is recommended but you can just as easily use the roaming app data to keep track of the same. There is also OneDrive SDK so you can upload the data to the cloud for better redundancy.


This isn’t ideal but I have provided feedback to the concerned product groups and I hope they are considering improvements for upcoming releases.


Thursday, January 30, 2014

Magic wand in Performance tuning - lazy instantiation

If you are in the business of programming for some time,  you likely know “the faster, the better” when it comes to the performance of the programming.

Take an example, in Microsoft store app certification requirement, it stated that app should response to any user action within 2 seconds in X86 or X64 and within 5 seconds in ARM architecture. and Microsoft has a certification verification kit to verify your app. Any app failed in this requirement would not get to the store.

You may also know that “The less is better” when it comes to line of code of the programming. if you can get the job done in 100 line, you would not want to make it to 110. The statistical study indicates that the more the lines of code, the more bugs it will be and the higher the maintained cost.

In most of cases, we programmers are fighting between these 2 goals. because in most of cases, faster performance means more code, but more code also mean longer execution time…  this is an art of balancing.

Take a look at this block of code in CustomerBO:

        Public Function GetCustomerDemographicInfo(ByVal customerID As Long, ByVal transactionID As Long, ByVal isTxnSummary As Boolean) As CustomerDemographicBE
            Dim custDemographic As CustomerDemographicBE = Nothing
            Dim customerDAO As CustomerDAO
            customerDAO = New CustomerDAO
            custDemographic = New CustomerDemographicBE
            custDemographic = customerDAO.GetCustomerDemographicInfo(customerID, transactionID, isTxnSummary)
            Return custDemographic
        End Function

CusotmerBO will surely interact with CustomerDAO quite often.  this looks like you are doing restaurant business and you hire one chef to cook one dish and fire him then hire another chef to cook another dish… There is lot of effort spent in hiring and firing. ( instantiation and  discard). In term of Line Of Code, you will see this line “Dim customerDAO As CustomerDAO = New CustomerDAO”  in almost every methods in the class.

As the rule says “ the less the better” and also out of laziness, you want to write less code, right? so you want to instantiate the class once and use it everywhere. so you thought of a perfect place to do this. “Constructor” . OK, you put this line of code in constructor and use it in all methods within the class.  The modified code will be similar to the following:

Public Sub New()
    mCustomerDAO = New CustomerDAO()
End Sub

and

Public Function GetCustomerDemographicInfo(ByVal customerID As Long, ByVal transactionID As Long, ByVal isTxnSummary As Boolean) As CustomerDemographicBE
            Dim custDemographic As CustomerDemographicBE = Nothing
            custDemographic = New CustomerDemographicBE
            custDemographic = mCustomerDAO.GetCustomerDemographicInfo(customerID, transactionID, isTxnSummary)
            Return custDemographic
End Function


If you do that in all methods within a class and do that for all BO classes, your total line of code will be around 5%  ( in the size of our code base total size of 800K, it likely mean 40K LOC reduction) and in most of case, your performance will be improved…

well, I made this statement is based on the assumption that you have a very good object model and there is in no time when an object is instantiated but never be used.  It turns out that this is a huge assumption to be made. In most of cases it is turns out to be a false assumption. This is especially the case  in our codebase. Take a look at this constructor :

    Public Sub New()
            mSelfAndMedicalCertificationBO = New SelfAndMedicalCertificationBO()
            mVehicleBO = New VehiclesBO
            mPlacardBO = New DisabilityPlacardBO()
            mDrivingLicenseBO = New DrivingLicenseBO()
            mTransactionDAO = New TransactionDAO()
            mPlateBO = New PlateBO
            customerDAO = New CustomerDAO()
            mDeficiencyDao = New DeficiencyDAO
            'objDrivingLicense = New DrivingLicenseDAO()
            mInterfaceBO = New InterfaceBO()
            mOrganDonorVoterRegistrationDAO = New OrganDonorVoterRegistrationDAO()
            mRegistrationBO = New RegistrationBO
            mCustomerBO = New CustomerBO()
            mOwnershipBO = New OwnerShipBO
            mTitleBO = New TitleBO
            mFlashDAO = New FlashDAO
            mFlashBO = New FlashBO
            mOfficeBO = New OfficeBO
            mPlateDAO = New PlateDAO
            mPaymentProcessingBO = New PaymentProcessingBO
            mRegistrationDAO = New RegistationDAO
            mDriverRecordSaleDAO = New DriverRecordSaleDAO
            mLienBO = New LienHolderBO
            mCustomerIDsCollector = New TwoKeyDictionary(Of String, Long, Long)
            mCommonReplicationBO = New CommonReplicationBO
            mOutputDocumentLibraryDAO = New OutputDocumentLibraryDAO
        End Sub



You might guess it right, it is for TransactionBO. for these who are familiar with our codebase, you know this class is a “knows all , does all” class. it handles all transactions of all types, from driver to vehicle, from living customer to deceased customer. from BAM transaction to legacy transaction; from interface to interactive…  put it in this way “ if you take this class out of the system, NOTHING will work.”  You can tell how big it is by looking at the LOC of this class, 16.7K!  It is as big as the size of a medium size system. it will never be a bad idea to break this class to smaller ones. For example, make one class handle Title transactions, and make another class handle DrivingLicense transactions . and only keep these common functionality  in TransactionBO and make it the base class for all others. Before we do that, we will see lots of objects been instantiated but never be used. ( what a waste!   from both performance and line of code point of views).

well, the short term fix could be lazy instantiate. (while I am proposing this temporary fix, I am still calling object structure redesign for TransactionBO, TransactionBPC and TransactionDAO, because the fundamental problems of a huge class handles all still exist and need to be dealt with.)

This is how to make it work.

1) remove substantiation statements in the construction for these objects may or may not be used in the class. ('mCustomerDAO = New CustomerDAO  or  Dim 'mCustomerDAO As New CustomerDAO  )
2) for these objects define another private member like this way:
Private _CustomerDAO As CustomerDAO
3) then change the original mCustomerDAO to make it a read only property

 Private ReadOnly Property mCustomerDAO() As CustomerDAO
            Get
                If _CustomerDAO Is Nothing Then
                    _CustomerDAO = New CustomerDAO()
                End If
                Return _CustomerDAO
            End Get
End Property

After that, all rest of the code stay as they are. That’s all it takes to implement lazy instantiation.

what is the performance implication ?

Before I recommend this pattern, I tried it with my POC, with this pattern, I managed to reduce response time of  my window store app from 15 seconds to less than 2 second when I only implemented it for one of UI class MediaElement. For my case, it is not that I instanced the object without using it. it is rather my strategy to distribute the substantiation time among different clicks. When User click on <lesson>  button to go to Lesson page, I hold off the instantiation of the MediaElement object within Lesson page. When user first time click on <Listen> button, I then  instantiate the MediaElement and then use the object make speech call.

when my  colleague  implemented  my recommendation in TransactionBPC , the performance test result is shown below:

Methods (before Fix) (after Fix)
TransactionBPC.GetOfficeHolidays() 12969 ms 1000 Iterations 3143 ms 1000 Iterations
TransactionBO.GetOfficeHolidays() 2136 ms 1000 Iterations 2177 ms 1000 Iterations
TransactionDAO.GetOfficeHolidays() 1922 ms 1000 Iterations 2108 ms 1000 Iterations

 The performance improvement in BPC and BO, DAO is more than 75%. while there is no noticeable change in BO or DAO


The reason I call it Magic Wand are
1) it does not take much effort, nor risk.
2) the pay back is great.
3) it is still a magic like getting a rabbit from a top hat. The real work of growing the rabbit still needed. like this case, to spend time to break the class to smaller ones is still a valid call from architecture point of view.


thanks and hope you enjoy your programming work as I do…

Announcement on Spelling Champion



Spelling Champion is published in Microsoft Windows Store for Windows 8.1 

click here to install the app 

click here to visit its web site




Monday, January 20, 2014

RE: VB.NET and C#

A friend of mine sent a link to a article listing down ten reasons on why VB.NET is better than C#. I attached that link in this post.

Whatever the author described about VB and C# is true,  but the conclusion is subjective.

Looking at the something or when you are facing the same description, you would get different conclusion when you look at it from different angle…

Thing as simple as “case sensitivity”  is not different, different angle will get different conclusion…

for a guys like me, was a VB6 expert and moved on to C#. I can appreciate both angles… what I want to say is  from VB to C#, it is a paradigm shift. Once you completed that shift you will be able to see both side.

It is true to C#  ( Java , or C++) programmer also. if the guy never try to understand the other side the fence. he or she would never understand why VB programmers think the way they do…

In VB.NET, If you define a property without specifying  type, it is fine;  if you have a function without return statement, it is fine; if you have function without return type, it is fine….

These in C# is unimaginable. you could say VB is more flexible ( relax)  than C#, you could also say C# is more rigid ( adjective) than VB. it is all depends on your prospect when you look at it.

VB.Net as it is now , good or bad, is due to the legacy it carried from VB. However, language is just a language, it is used to describe our thoughts only. it is our thought that matters.
  




Bad Smell in code Refused Bequeast

This is what the book says about "Refused Bequest"

Subclasses get to inherit the members and data of their parents ( base class). but what if they don't want or need what they are given? they are given all these great gifts and pick just a few to play with.
The traditional story is that this means the hierarchy is wrong. You need to create a new sibling class and use "Push Down" these unused members to the sibling. This way the parent ( base class) holds only what is common. Often you will hear advices that all superclass (base classes) should be abstract.
You'll guess from our snide use of traditional that we are not going to advice this, at least not all the time. We do sub-classing to reuse a bit of behaviors all the time, and we find it is perfectly good way of doing business. There is a smell, we can't deny, but usually it isn't a strong smell. so we say that if the refused bequests is caused confusion and problem, follow the traditional advice. However, don't feel you have to do it all the time. Nine times out of ten this smell is too faint to be worth cleaning.
The smell of refused bequest is much stronger if the subclass is reusing behavior but does not want to support the interface of the supper-class. We don't mind refusing implementations, but refusing interface gets us on our high horses. In this case, however, don't fiddle with the hierarchy; you want to gut it by applying "Replace Inheritance with Delegation.

.
From here, the author identified two bad smells, indicated one is worse than the other. The less sever one is "refusing members from the base class" , the more serve one is "refusing interface from the base class or interface". Let me touch on them in detail in reference to our codebase, less sever one first.


The book says "if refused bequests is caused confusion and problem, follow the traditional advice" so I am going to repeat the advice here: If they cause confusion, push them down to where it should be, create middle level classes if need to. the guidelines on this principle is 1) define one specific business logic or data element only in one place, 2) do not provide members to the class that does not make sense or not needed. I was made to believe some of these subclasses or members are not in use... if so we can safely remove them from our code base.

Refusing interface from the base class or interface

it means if your class implement your own interface, you want to provide implementations for all its members.
if you do not want to implement certain interface, you either throw Not Implemented exception to notify consumers or provide default implementation, or remove the member form the interface.

However, the solution is simple:

1) if the class or a member is no longer in use, take them out, if you are not 100% sure, comment them out or exclude them form the project but keep them in the version control library, in case we want to take them back..
2) if the member in the interface is no longer needed, take it out and talk it out from all it implementations
3) If the public member is no longer needed, take it out and all remove reference to it.
4) For private methods if you might want to put the logic in it later, at least you want to put a todo in it, so that people knows what is going on here.



This is like you live in your own houses, you want to take out the trash regular, you want to do some clean up once a while, you want to put the right thing in the right place once a while. this is so that you will enjoy your living more. Imagining in a house, for years, nobody does these things. There are trash everywhere, stuff are laying down all over the places. empty milk jar, leftover food are in the fridge. Do you enjoy living in the house, can you be productivity in the house? Sure enough, there are times, we are in the hurry, so we leave the dirty dish in the sink and heading out for something urgent. but when we are back home, or in the evening or weekend, we want to clean them up. The same is true, in software development, there are times ( a quick fix before it is released, or must fix a production defect in quickest possible way.) we leave something behind us, but we need to clean them up after the urgent issue is addressed. These are not some rocket science, it has nothing to do with technology. They are some basic practices in software engineering. however, a good deed goes long way down the road. Sometimes it might as important of "make it or break it". in a software development project.

Let's work together to make our project a successful one and a enjoyable one.

Friday, January 10, 2014

Securing ASp.Net Web API Service

Before Christmas I wrote an article on developing Services for Modern Application with ASP.Net Web Today, I want to pick up the topic again go deeper on making our service closer to production quality service.. Let’s get it started…

If you walked through my article and tried to create a Web API service, you will find it is almost effortless. well, there is a say goes this way “If it is too good to be true, probably it is” I find there are some wisdom in it.  the server we created is next to useless in production environment.  please hear me out on why I said that.

Imagine you put up this service in production on Internet, sure enough , your app can access it, but so does anybody on the internet!  and more over, that is a host of search engines like google, bing. They are working day and night sniff around on the internet. Do you mind they can hit your service to suck the data out from your system? do you want someone on the internet rage some kind of DOS attach on your site?

I am sure, you do not want any of these happen to you?  

so an unprotected site is never be good enough for live operation. The question put in front of us how to secure our Web API site?

Thanks to Microsoft, the solution is provided in Visual Studio.  When you start a new Web API project, you will see this screen:




If you are like me when I worked on it the first time, you would type the project name and click <Ok>. but slow down a bit, let’s take a closer look at this screen. on the low right portion of the screen, you see Authentication  No Authentication, and there is a button on top of it with the caption “Change Authentication”. let’s click on the button to see what we can get.

wow, it offers 4 different portions:

No Authentication :
default
Individual User Accounts:
for applications that store user profile in a SQL Server database, Users can register, or sign in using their existing account for Facebook, Twitter, Google, Microsoft or another provider.
Organization Accounts
for applications that authenticate users with AD, Window Azure AD or Office 365
Window Authentication
for Intranet Application

For an enterprise system, you most likely will go with  Organization Accounts or Window Authentication. For public facing application like ones in the Apple store, Google store or Microsoft store, you probably want to go with Individual User Accounts.

Let’s say we go with Individual Accounts.

You select Individual Accounts and clock on Ok button, then the IDE create the project for you as it did before. nothing special.

However, after the project is created you run your site and try to hit the site by http;//localhost://3593/api/Values

you will get “Authorization has been denied for this request”

if you go and take a closer look at the code the IDE generated, you will find 2 differences. one is there is attribute on the controller class

[Authorize]
public class ValuesController : ApiController


another one is you will see a new controller shown below:

[Authorize]
public class AccountController : ApiController


Another difference is not obvious, but very significant, that is the site come with a full function OAuth2 service implementation.  If you are interested, you can search for OAuth2 on the web, you can get tons of information….

That means the site is protected! That ‘s a good sign, but how to access it?  If you are doing “single Page “ Web project,  Microsoft gives you all the code to access the site as part of project template, all you need to do is provide the site base address. If you are working on Windows Store app or desktop app you need to code yourself.  Leaving the easy thing aside, let me touch on the details in Windows Store App / desktop app.   The code I am going to show you here are from Windows Store App, but it will be same if it is form a desktop app, basically, it access the site through HttpClient class.

In order to access the site, you need to have an account on the site, you can register an account with the site by hitting the AccountController.


the following code is used to register an  internal account

  public async Task<bool> Register(AccountRegistation registation)
       {
            Client = new HttpClient();
            Client.BaseAddress = new Uri(baseAddress);
            Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
           HttpResponseMessage response = await Client.PostAsJsonAsync<AccountRegistation>( "api/Account/Register", registation);

           return response.IsSuccessStatusCode;
       }

and AccountRegistation is a data class show below:

public  class AccountRegistation
    {
       public string UserName { get; set; }
       public string Password { get; set; }
       public string ConfirmPassword { get; set; }

    }


once you register an account with the site, you can  hit the OAuth2 service to get security token, the code for that is similar to the code below:

   public async Task<BearerToken> RequestToken(Credential credential)
       {
           var response = await Client.PostAsync(Token, credential.StringContent);
           var tokenResponse =   await response.Content.ReadAsAsync<BearerToken>();
           return tokenResponse;
       }

where  BearerToken and   Credential are the data class shown below :

    public class BearerToken
    {
        [DataMember(Name = "access_token")]
        public string AccessToken { get; set; }

        [DataMember(Name = "token_type")]
        public string TokenType { get; set; }

        [DataMember(Name = "expires_in")]
        public int ExpiresIn { get; set; }

        [DataMember(Name = "userName")]
        public string UserName { get; set; }

        [DataMember(Name = ".issued")]
        public DateTime Issued { get; set; }

         [DataMember(Name = ".expires")]
        public DateTime Expires { get; set; }
    }


public class Credential
    {
       public GrantType GrantType { get; set; }
       public string UserName { get; set; }
       public string Password { get; set; }
    }



Once you have the token, you can put the token on the request before you hit the Values controller. the code for that is similar to the following:

  token = await SpellingServiceProxy.RequestBearerToken(TestSettings.OAuthServiceBaseAddress);

         SpellingServiceProxy proxy = new SpellingServiceProxy("http://localhost:1234", token);
         IList<Product> result = await proxy.RetrieveCatalogAsync();


This is just introduction. OAuth 2 is a big topic by itself.  Account Controller could be another topic by itself. With Account Controller you can do the following:

Register internal user account
Register External user account
login and logout
manage account info
Change password
Set password
Add External Login
Remove Login
ExternalLogin / Logout

Monday, January 6, 2014

SpeechSynthesizer throws exception in windows 8.1 computer and RT 8.1 device

In my development lab, I have 2 windows 8.1 computers and one first generation Surface device. One of the Windows 8.1 box was a fresh windows 8.1 installation, another one was upgraded from Window 8.0 to Window 8.1.
I use them for mt TTS app development Well, a few days ago, I got a new Surface 2 device. I was so excited about it. I wasted no time before loading my TTS app to it. To my surprise, as soon as the code is supposed to talk,  the app crashes. I feel I have the obligation to figure what is going on and how to bring fix it in Surface 2 before me submitting the app to Microsoft app store. My first action was going back to the Speech synthesis sample provided as part of SDK library and run the sample app. I got the same behavior I filed a support ticket with Microsoft, have been working with Microsoft support professional for about a week. During the time, the same behavior starts to show up in my fresh window 8.1 box.  After many late nights and a chain of emails back and forth, the root cause is identified and walk around is proven working. This turns out to be a bug in Microsoft 8.1 in both RT and Windows OS. The bug only shows up in fresh installed 8.1.

As soon as this line is executed:

synthesisStream = await this.synthesizer.SynthesizeSsmlToStreamAsync(MyTextToBeRead.Text);  

An exception is thrown
               A first chance exception of type
'System.UnauthorizedAccessException' occurred in mscorlib.dll

WinRT information: Access is denied.

The root of the issue seems to be with the permissions on CurrentUserLexicon key under HKEY_CURRENT_USER\Software\Microsoft\Speech. If you run following powershell command on the problem box, you will get the result shown on the screenshot:
Get-Acl -Path HKCU:"Software\Microsoft\Speech\CurrentUserLexicon" |Format-List



The problem is with the extra entry of “ALL APPLICATION PACKAGES”. This extra entry only exists in fresh installed Window 8.1. All it takes is remove this permission entry from Software\Microsoft\Speech\CurrentUserLexicon
After remove the permission entry from the registry key, Under Regedit you will get following screenshot:


Under window powershell, you will get screen shot similar to the following:


Once that is done, your SpeechSynthesizer will start talking.


If you upgrade from Windows 8.0 to Windows 8.1 you are fine. That explains why it works in Microsoft Surface but not in Surface 2, because Surface 2 is a fresh installed Window RT 8.1, while in Surface case, it is upgraded from Windows RT 8.0 to Window RT 8.1.