Category Archives: Architecture

Composing a Client in a Client/Server Architecture Model without Service

I will start by saying this:

If all you have is the functional requirements of the client application and known table structure of the underlying database and the service interface has not yet been established, then let the requirements of the client application drive the service contract.  The structure of the underlying database should not drive this service contract.

Over the course of my career, I have had to architect several solutions against external systems, mostly webservices, which either had not been developed nor had not yet been conceptualized or were in the initial stages of development.

Recently at Phreesia, we had to develop against a web service which was in its initial stages of conception.   At Honeywell’s Global Tracking division , when we migrated the legacy OCC 200, a 20+ years old legacy VMS Search and Rescue application, to the OCC 600, a modern client/server architecture, there was enormous amounts of effort expended in formulating the functional requirements of the desktop application since getting the user interface and all its usability concerns right was of utmost concern to facilitate adoption.

While such efforts was spent on defining the requirements of the client, the requirements of the server were not being developed at the same pace.  Consequently, development of the client started well in advance of the server and in such a case, a decision had to be made on how to compose the Data Access Layer of this desktop application.

We started development of the client side WPF desktop application, by decomposing the application into the three main layers: Presentation Layer Tier, Business Service Layer, Data Access Layer as shown below:

three layered application.png

We had a solid set of functional requirements against the Presentation Layer, which also drove the Middle Layer as well as the DAL. We also had an understanding of the database even though this was not documented and were able to work with the back-end developers to ensure that the contracts we were formulating could be met.  Based on the nature of the application’s use case in Search and Rescue; we are talking about a heavy desktop GIS mapping application, significant amount of functionality was developed before the Web Service came online, creating fake Web Services which implemented the same service contracts.

The Data Access Layer

As the saying goes, this is where the rubber hits the road,  this layer was responsible for retrieving data from the underlying source of truth, our databases and making this available to the application.  It’s purpose was two fold:

  1. Manage all connectivity with the underlying data source:  This was achieved via WCF service constructs.
  2. Encapsulate data retrieval from the application: This was achieved via interfaces and dependency injection.
  3. Map external entities retrieved from the underlying data source to business objects are required by the application: This was achieved via the Service Oriented Architectural and Chain of Responsibility pattern.

This layer was a dedicated set of DLLs which were injected into the application via IoC allowing the front and backend development to continue in a loosely coupled fashion.  The application’s DAL ended up looking like this:

DAL decomposition (1).png

The experiment was a success, after which we had a highly usable and performance WPF desktop application which the end customer has grown to like and adopt.

In another blog, post I will address the issue of mapping from web service entities to the application domain model.  How we did this in a consistent, loosely coupled, testable and extensible manner.

What has been your experiences?

Advertisements

Beware of changing variable names in SOAP Webservices

When improving upon your SOAP webservices currently in production, it might occur to you to change variable names in method signatures to make the more readable and conform to your coding practices. After these are the small incremental improvements we make to reduce our technical debt. But before doing this, please beware of the ramifications.

A WSDL describing a SOAP webservice is “strongly typed” in the sense that method signatures including method names, variable names and their types, once defined, cannot be changed with breaking backward compatibility. And Since SOAP does not provide an easy way for clients to detect an API change, altering either of the constructs that define your WSDL guarantees a temporal disruption of service due to your current API consumers.

For example, say you have a SOAP webservice exposing this method:

public Report GetReport(string ReportName)

which is defined as part of your WSDL. If you later change this this method to say

public Report GetReport(string reportName)

the service will receive null as the reportName in some client implementations. Others may simply crash since this endpoint is no longer known. If one is spending quite some time working with SOAP based services on a daily basis, it becomes almost of a second nature to regenerate the client proxy based on the published WSDL to ensure consistency otherwise, you get these strange behaviors. However, if you are spending a lot of time in RESTFul services, chances are you are not worrying about these types of problems. In this case, there is a good chance you will rename a webservice API parameter name in the name of improving readability (nothing wrong with that) only to find out that the next morning none of your clients are able to connect to your API.

And no, unit tests will not catch this.

A simple cross platform status bar micro service

These days, with all the buzz and best practices around micro-services, Clean Code and KISS, one should be proud of themselves after creating a simple, small and cross platform micro-service to further advocate the power of these patterns and demonstrate how easy developing robust, maintainable and beautiful code can be. Any complex problem can be broken down into small manageable chunks.

Requirement was to add a status bar to a WPF application to indicate progress of some background calculations. I wanted to use MVVM and also wanted to be able to access the main application’s status bar from anywhere in the application, mostly from the ViewModels. ViewModels are currently being injected services from the Unity container so I thought, hey, why not create a simple StatusBarService that can be bound to a StatusBar control, which the ViewModel can the control from there? I did something similar year ago but this approach is more mature and greatly simplified.

The service in its entirety as follows:

 public class StatusBarService : IStatusBarService, INotifyPropertyChanged
    {
        public StatusBarService()
        {
            ProgressBarVisibility = Visibility.Hidden;
            Name = null;
            Message = "Ready";
        }

        public string Name { get; private set; }

        private bool _isIndeterminate;
        public bool IsIndeterminate
        {
            get { return _isIndeterminate; }
            set
            {
                _isIndeterminate = value;
                OnPropertyChanged();

                if (!_isIndeterminate)
                {
                    ProgressBarVisibility = Visibility.Hidden;
                }
                else
                {
                    ProgressBarVisibility = Visibility.Visible;
                }
            }
        }

        public void StartIsIndeterminate(string message)
        {
            IsIndeterminate = true;
            Message = message;
        }

        public void StopIsIndeterminate(string message)
        {
            IsIndeterminate = false;
            Message = message;
        }

        private string _message;
        public string Message
        {
            get { return _message; }
            set
            {
                _message = value;
                OnPropertyChanged();
            }
        }

        private Visibility _progressBarVisibility;
        public Visibility ProgressBarVisibility
        {
            get { return _progressBarVisibility; }
            set
            {
                _progressBarVisibility = value;
                OnPropertyChanged();
            }
        }

        public event PropertyChangedEventHandler PropertyChanged;

        [NotifyPropertyChangedInvocator]
        protected virtual void OnPropertyChanged([CallerMemberName] string propertyName = null)
        {
            PropertyChangedEventHandler handler = PropertyChanged;
            if (handler != null) handler(this, new PropertyChangedEventArgs(propertyName));
        }
    }

It is cross platform and does not care how you use it.

You can register it with your IoC container and modify your ViewModel constructors to be injected an instance of IStatusBarService. Once you have access in your ViewModel, you can control any Statusbar as you see fit. Only caveat is ensuring the StatusBarService is bound to the DataContext of the StatusBar it should be controlling.

Happy Coding.

Factories or Providers: what is your chosen terminology?

The Factory pattern in plain language is about encapsulating an entity that is responsible for creating stuff provided with a given context. For example, say I wanted to create such an entity responsible for created different implementations of an ILoggerFacade interface defined as such:


public interface ILoggerFacade
{
void Error(object error);
void Info(object info);
}

I could define an ILoggerFacadeFactory interface as follows:


public interface ILoggerFacadeFactory
{
ILoggerFacade CreateLogger(object owner);
}

and have an implementation for Console, log4Net or other logging libraries as follows:

public class ConsoleLoggerFacadeFactory : ILoggerFacadeFactory
{
public ILoggerFacade CreateLogger(object owner)
{
return new ConsoleLoggerFacade(owner);
}
}


public class Log4NetLoggerFacadeFactory : ILoggerFacadeFactory
{
public ILoggerFacade CreateLogger(object owner)
{
return new Log4NetLoggerFacade(owner);
}
}

Then this becomes a matter of registering the appropriate factory in an IoC container, such as Unity, to create the logger as desired:

   ioc.Register<ILoggerFacadeFactory, Log4NetLoggerFacadeFactory>();

This way, I can easily change the type of logger I use in my application via configuration. I use this pattern extensively, especially with 3rd party libraries. I have used this to abstract message queue implementations for MSMQ, RabbitMQ and ZeroMQ, logging and even for UI elements such as DataGrids, DataCharts and Docking containers, without thinking much about the nomenclature.

Recently, however, this terminology has caused some excitement in my professional circles. Others will rather call these types of entity factories, entity providers for good cause. After all, a provider provides stuff. Factories or Providers sort of provide similar responsibilities even though I would argue that factories create stuff while providers do not necessary do so. Either way, there is no right or wrong answer here although I tend to lean toward calling entity creators, factories, for the very reason that the create stuff by typically calling some sort of constructor on the object class.

Just like an automobile factory manufactures cars, an entity factory instantiates new objects. An automotive dealer, on the otherhand, is an auto provider but cannot be considered an automobile factory. This is the way I distinguish these responsibilities. In fact, a provider can be given reference to a factory that will create stuff that is requested of it, it not in its inventory. Distinct responsibilities.

This is the way I see it. What are your thoughts?

Some defensive coding practices against environmetal uncertainties – part 1

1. Always consider Environment.CurrentDirectory as opposed to Assembly.GetExecutingAssembly().Location, when getting current working directory.

This innocent looking line of code causes grief.

var pathToSqliteDatabase = Path.Combine(Assembly.GetExecutingAssembly().Location, "SqliteDb.db3");

Why?

Let’s just first ignore the fact that a test harness should be going directly to a database since this is part of the problem. But if you are running a test harness that has to resolve a location to an SQLite database instance you will get interesting results.

If using MSTest, the above line of code resolves to the current test project’s working folder. So if your test project was in c:\dev\SQliteAdventures, pathToSqliteDatabase to be

c:\dev\SQliteAdventures\bin\Debug

However, if you decided to change to NUnit and have ReShaper installed, the same above line of code now resolves to some random location:

In my case, it looked like this:

C:\Users\knji\AppData\Local\Temp\jahvgmug.pjb\s5hvswgs.kil\SQliteAdventures\assembly\dl3\086a6c5d\1860476f_4375d001\

Changing the above line of code to

var pathToSqliteDatabase = Path.Combine(Environment.CurrentDirectory, "SqliteDb.db3");

will return the current result regardless of the operating environment. So this is preferable here.

2. Carefully consider using x86 vs x64 versions of 3rd party dependencies.

The System.Data.SQLite.dll .NET connector for SQLite, for example comes, with an x86 and x64 version. When executing tests using MSTest, x86 version of the is required since the MStest runner executes within the 32 bit Visual Studio process. However, if you are running NUnit via ReSharper, which executes within its own 64 bit process called JetBrains.ReSharper.TaskRunner.CLR45.x64.exe, you need the x64 bit version of this dll otherwise you get image loading exception. Besides testing against abstractions or fakes, I do not yet know to automate this. Ideas always welcomed.

To be continued…..

Abstracting message queue creation using the factory pattern

You had to create several additional modules for an existing enterprise solution but did not know which of the various message queues to use: your options are MSMQ, ZeroMQ, RabbitMQ, Azure, etc.
You decided to postpone this decision to latter in the game but in the meantime, abstracted away functionality of the message queue behind an interface that looks like this:

  public interface IMessageQueue : IDisposable
    {
        string Address { get; }
        IDictionary<string, string> Properties { get; }
        void InitializeInboundQueue(MessageQueueCreationContext creationContext);
        void InitializeOutboundQueue(MessageQueueCreationContext creationContext);
        void Send(Message message, string messageTopic = null);
        void Listen(Action<Message> onMessageReceived, string routingKey = null);
        void Receive(Action<Message> onMessageReceived);
        IMessageQueue GetResponseQueue();
        IMessageQueue GetReplyQueue(Message message);
    }

The creation context class looks like this:

  [Serializable]
    public class MessageQueueSpecs : IEquatable<MessageQueueSpecs>
    {
        public bool Equals(MessageQueueSpecs other)
        {
            if (ReferenceEquals(null, other)) return false;
            if (ReferenceEquals(this, other)) return true;
            return 
                string.Equals(Name, other.Name) && 
                string.Equals(Address, other.Address) && 
                MessagePattern == other.MessagePattern && 
                Direction == other.Direction && 
                string.Equals(MessageTopic, other.MessageTopic) && 
                string.Equals(HostName, other.HostName);
        }

        public override int GetHashCode()
        {
            unchecked
            {
                var hashCode = (Name != null ? Name.GetHashCode() : 0);
                hashCode = (hashCode*397) ^ (Address != null ? Address.GetHashCode() : 0);
                hashCode = (hashCode*397) ^ (int) MessagePattern;
                hashCode = (hashCode*397) ^ (int) Direction;
                hashCode = (hashCode*397) ^ (MessageTopic != null ? MessageTopic.GetHashCode() : 0);
                hashCode = (hashCode*397) ^ (HostName != null ? HostName.GetHashCode() : 0);
                return hashCode;
            }
        }

        public static bool operator ==(MessageQueueSpecs left, MessageQueueSpecs right)
        {
            return Equals(left, right);
        }

        public static bool operator !=(MessageQueueSpecs left, MessageQueueSpecs right)
        {
            return !Equals(left, right);
        }

        public MessageQueueSpecs()
        {
            AdditionalProperties = new Dictionary<string, string>(5);
            Name = GetType().FullName + DateTime.UtcNow;
        }

        public string Name { get; set; }
        public string Address { get; set; }
        public MessagePattern MessagePattern { get; set; }
        public Direction Direction { get; set; }
        public ISerializer Serializer { get; set; }
        public IDictionary<string, string> AdditionalProperties { get; set; }
        public string MessageTopic { get; set; }
        public string HostName { get; set; }

        public bool IsValid()
        {
            if (Serializer == null)
                return false;

            return true;
        }

        public override bool Equals(object obj)
        {
            if (ReferenceEquals(null, obj)) return false;
            if (ReferenceEquals(this, obj)) return true;
            if (obj.GetType() != this.GetType()) return false;
            return Equals((MessageQueueSpecs) obj);
        }

        public string GetUid()
        {
            return typeof (MessageQueueSpecs) + Name + Direction + GetHashCode();
        }

To delegate the creation of message queues to an independent entity that can be passed to stakeholders via IoC, we define an IMessageQueueFactory as listed below:

 
  public interface IMessageQueueFactory
    {
        IMessageQueue CreateInboundQueue(MessageQueueSpecs specs);
        IMessageQueue CreateOutnboundQueue(MessageQueueSpecs specs);
    }

The interface can possibly be simplified using only one method but two methods gives the API clarity and also gives us the flexibility to define different parameters for inbound or outbound queues among others, possibly segregating the interface if need be.

The implementation of the IMessageQueueFactory depends on how many different types of message queues we want to support in the solution. To keep things simple, we will assume that only one type of message queue will be used in this solution and the one we choose is RabbitMQ.

The implementation of IMessageQueueFactory for RabbitMQ is as follows:

  public class RabbitMessageQueueFactory : IMessageQueueFactory
    {
        private readonly MemoryCache _cache = new MemoryCache("RabbitMessageQueueFactory");
        
        public IMessageQueue CreateInboundQueue(MessageQueueSpecs specs)
        {
            return GetOrCreateQueue(specs);
        }
        
        public IMessageQueue CreateOutnboundQueue(MessageQueueSpecs specs)
        {
            return GetOrCreateQueue(specs);
        }

        private IMessageQueue GetOrCreateQueue(MessageQueueSpecs specs)
        {
            var cacheKey = specs.GetUid();
            var queueObj = _cache.Get(cacheKey);
            if (queueObj != null)
                return (IMessageQueue) queueObj;

            var queue = new RabbitMessageQueue(specs);
            _cache.Add(cacheKey, queue, new CacheItemPolicy());

            return queue;
        }
    }

Creating inbound and outbound RabbitMQ queues follow a similar pattern based on our message queue implementation so construction is easy. To use our factory, we can register it with an IoC container such as Unity or AutoFac or create an instance and pass it to interested parties.

For the sake of completeness, RabbitMessageQueue and MessageQueueBase is implemented as follows:


   public sealed class RabbitMessageQueue : MessageQueueBase
    {
        private readonly ISerializer _serializer;
        private IConnection _connection;
        private IModel _channel;
        private QueueingBasicConsumer _consumer;

        public string ExchangeType
        {
            get { return GetProperty(ExchangeTypePropertyName, "topic"); }
        }

        public string Exchange
        {
            get { return GetProperty(ExchangePropertyName, "topic"); }
        }

        public RabbitMessageQueue(MessageQueueSpecs specs)
        {
            _serializer = specs.Serializer;
        
            CopyProperties(specs.AdditionalProperties);
            ReconcileProperties(specs);
            InitializeInboundQueue(specs);
            InitializeOutboundQueue(specs);
        }
        
        private void ReconcileProperties(MessageQueueSpecs specs)
        {
            ReconcileProperty(MessageTopicPropertyName, () => specs.MessageTopic);
            ReconcileProperty(ExchangePropertyName, () => specs.Address);
            ReconcileProperty(ExchangeTypePropertyName, () => specs.MessagePattern == MessagePattern.PublishSubscribe ? "topic" : "fanout");
        }

        private void CopyProperties(IEnumerable<KeyValuePair<string, string>> properties)
        {
            if (properties != null)
            {
                foreach (var property in properties)
                {
                    AddProperty(property.Key, property.Value);
                }
            }
        }

        protected override void Dispose(bool disposing)
        {
            if (disposing && _channel != null)
            {
                _channel.Dispose();
                _connection.Dispose();
            }
        }

        protected override string GetAddress(string queueName)
        {
            throw new NotImplementedException();
        }

        private void CreateInboundQueue(MessageQueueSpecs specs)
        {
            var exchange = CreateChannel(specs);

            //2. create a non-durable, exclusive, autodelete queue with a generated name
            var result = _channel.QueueDeclare();
            var routingKey = specs.MessageTopic ?? string.Empty;
            //3. bind to the exclusive queue created above
            _channel.QueueBind(result.QueueName, exchange, routingKey);

            //4. now get message from queue
            _consumer = new QueueingBasicConsumer(_channel);
            _channel.BasicConsume(result.QueueName, true, _consumer);
        }

        private string CreateChannel(MessageQueueSpecs specs)
        {
            var hostName = specs.HostName ?? GetProperty(HostNamePropertyName, "localhost");
            var factory = new ConnectionFactory { HostName = hostName };
               
            _connection = factory.CreateConnection();
            _channel = _connection.CreateModel();

            _channel.ExchangeDeclare(Exchange, ExchangeType); 
            return Exchange;
        }

        private void CreateOutboundQueue(MessageQueueSpecs specs)
        {
            CreateChannel(specs);
        }

        public override void InitializeInboundQueue(MessageQueueSpecs specs)
        {
            Initialize(Direction.Inbound, specs.Address, specs.MessagePattern);
            // NOTE: for fanout, it apears as though we can use the same initializations defined in CreateInboundQueue
            CreateInboundQueue(specs);
        }

        public override void InitializeOutboundQueue(MessageQueueSpecs specs)
        {
            Initialize(Direction.Outbound, specs.Address, specs.MessagePattern);
            // NOTE: for fanout, it apears as though we can use the same initializations defined in CreateInboundQueue
            CreateOutboundQueue(specs);
        }

        public override void Send<T>(T message, string messageTopic = null)
        {
            var body = _serializer.MessageToBytes<T>(message);
            var exchange = GetProperty(ExchangePropertyName, Name);// "authentictions";
            if (string.IsNullOrEmpty(messageTopic))
                messageTopic = GetProperty(MessageTopicPropertyName, string.Empty);

            _channel.BasicPublish(exchange, messageTopic, null, body);
        }
    
        public override void Receive<T>(Action<T> onMessageReceived)
        {
            var ea = _consumer.Queue.Dequeue();
            var body = ea.Body;
            onMessageReceived.Invoke(_serializer.BytesToMessage<T>(body));
        }

        public override IMessageQueue GetResponseQueue()
        {
            throw new NotImplementedException();
        }

        public override IMessageQueue GetReplyQueue(Message message)
        {
            throw new NotImplementedException();
        }

        public override string ToString()
        {
            var properties = Properties;
            if (properties == null)
            {
                properties = new Dictionary<string, string>();
            }

            var propertyInfo = GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public);
            foreach (var pinfo in propertyInfo)
            {
                if (properties.ContainsKey(pinfo.Name)) continue;

                var value = pinfo.GetValue(this);
                properties.Add(pinfo.Name, value != null ? value.ToString() : string.Empty);    
            }

            var sb = new StringBuilder(100);
            foreach (var kvp in properties)
            {
                sb.AppendLine(string.Format("\t {0}:{1}", kvp.Key, kvp.Value));
            }
            return sb.ToString();
        }
    }

public abstract class MessageQueueBase : IMessageQueue
    {
        public const string ExchangePropertyName = "Exchange";
        public const string ExchangeTypePropertyName = "ExchangeType";
        public const string MessageTopicPropertyName = "MessageTopic";
        public const string HostNamePropertyName = "HostName";

        private readonly IDictionary<string, string> _properties = new Dictionary<string, string>(10);

        protected void Initialize(Direction direction, string name, MessagePattern pattern)
        {
            Name = name;
            Direction = direction;
            MessagePattern = pattern;
           
        }

        protected void ReconcileProperty(string name,Func<string> valueFactory)
        {
            if (!Properties.ContainsKey(name))
            {
                Properties.Add(name,  valueFactory.Invoke());
            }
        }

        public void AddProperty(string name, string value)
        {
            _properties.Add(name, value);
        }

        public string GetProperty(string propertyName, string defaultValue)
        {
            if (_properties.ContainsKey(propertyName))
                return _properties[propertyName];

            return defaultValue;
        }

        public string Name { get; private set; }
        public Direction Direction { get; private set; }
        public string Address { get; private set; }
        public MessagePattern MessagePattern { get; private set; }

        public IDictionary<string, string> Properties
        {
            get { return _properties; }
        }

        public abstract void InitializeInboundQueue(MessageQueueSpecs specs);

        public abstract void InitializeOutboundQueue(MessageQueueSpecs specs);

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected abstract void Dispose(bool disposing);
        protected abstract string GetAddress(string queueName);

        public void Send(Message message, string messageTopic = null)
        {
            Send<Message>(message, messageTopic);
        }

        public abstract void Send<T>(T message, string messageTopic = null) where T : class;

        public void Listen(Action<Message> onMessageReceived, string routingKey = null)
        {
            Listen<Message>(onMessageReceived, routingKey);
        }

        public virtual void Listen<T>(Action<T> onMessageReceived, string routingKey = null) where T : class
        {
            Console.WriteLine(" [*] Waiting for messages." +
                             "To exit press CTRL+C");

            while (true)
            {
                Receive(onMessageReceived);
            }
        }

        public abstract void Receive<T>(Action<T> onMessageReceived);

        public void Receive(Action<Message> onMessageReceived)
        {
            Receive<Message>(onMessageReceived);
        }
        public abstract IMessageQueue GetResponseQueue();
        public abstract IMessageQueue GetReplyQueue(Message message);
    }

Happy coding!

How do you organize core projects in your .NET solution

There has always been a constant struggle on how to organize core or infrastructural projects in my .NET solutions. See, I started back in the day of the Composite Application Block (CAB) and with the early version of the Microsoft Enterprise Libraries from the Patterns and Practices group. Ten years ago when you created a desktop solution using the CAB template you had an organizational structure that contained infrastructural projects like so:

infrastructure layout

With modules looking like this:

These modules use services defined in Infrastructure.Services but via dependency injection. Over the years, this pattern has influenced the structure of my past and current solutions.

While I do like the pattern, there has always been a persistent challenge of determining what goes where? The original premise of Infrastructure.Dictionary was to contain all business entity definitions or what we we call the business model. Infrastructure.Interface was supposed to contain service interfaces the idea being that these two projects together define the system API. What ended up happening as the solution became bigger is interfaces and UI elements started creeping into Infrastructure.Dictionary.

Recently, I decided to rethink this approach of having two separate projects:

All modules that reference Infrastructure.Interface, must also reference Infrastructure.Dictionary. In a solution containing 30 projects, there is a single project that references Instructure.Dictionary and not Infrastructure.Interface. This project which contains symbology editors for points, lines and polygons cannot be used deployed on its own,therefore, it must be deployed within the same process space in which Infrastructure.Dictionary has been loaded. Does it make sense to continue separating Infrastruture.Dictionary and Infrastructure.Interface projects. Should I just have a single project called Infrastructure.Core?

What are others doing?