Category Archives: IoT

Projecting how 5G will impact operations

It’s coming they say.    Not winter (that’s here already), but mobile 5G is coming.   Sure the latency will allow a new generation of mobility applications.   The new RF control functions will allow better elasticity.   The network slicing will finally allow MPLS-like functionality in a 3GPP network.    But I think its time someone asked the basic questions on how 5G will impact operations.

Projecting how 5G will impact operations

Basic Tutorial and Terminology

5G means different things to different people.   The money involve means industry will be providing competing visions.   The essence of 5G is focusing on improving latency, bandwidth, network slicing, and elasticity.   Adding more endpoint (antennas) and distributed control functions will reduce the latency.    The control functions permitted more distributed switching.   The extra endpoints means fewer network mileage required.   These latency benefits are the biggest game changer so far.

The simplified control functions also enable better elasticity options.   Scaling up/down/in/out will allow a more natural self-optimizing network.   Adding class-of-service capabilities (like QoS in MPLS) will allowed tiered network options.   While net-neutrality questions still loom, this adds diversity to the single-use mobile network.   The amount of network required is still to be defined, but it looks to be at least 10x.   Mobile operators are also taking this opportune time to diversify their vendors.    Most US providers are adopting radios from at least two vendors.

The bottom line for operations is potentially terrifying.   Exponential scale in the network and backhaul is to be expected.    Exponential complexity increase with a “always in flux” network.   New network offerings and customer bases are bound to cause trouble.   Top those off with at least doubling the vendors.   Houston we have a problem!


Projecting how 5G will impact operations

Next-generation Mobile Network Services

As you can see, the investment will be significant.   The upside should be worth it though.   Traditional mobile services are commoditized as I blogged here last year.   Data, voice, and SMS do not provide enough value to the customer.  The new services provided will change that.   With the latency benefits, IoT services will become more viable.   I detailed IoT more in the previous blog around IoT Service Assurance.   In my opinion, the most intriguing new offering is “fixed wireless access” (FWA).    Ericsson did a really nice write up available here.    Verizon is augmenting their FIOS offer with a FWA offer in 2018.   This means that mobile providers are entering into the cable access market (HFC).    This sets Verizon against Comcast or T-mobile against Charter.   Gone will be the days that we will locked into high speed internet options solely by developed the neighborhood.

These new network services will drive new revenue potentials.   Most of these services will have direct competition so quality will matter.   With all these changes operations should expect challenges.    We should all expected quality problems with these new services.


Projecting how 5G will impact operations

Exponential Scale and Complexity

The first great challenge will be scale and complexity.  Tripling the number of devices in your network will stress your tools.    Realistically, can your OSS handle a 10x-1,000x increase in network size?    But this is not the only issue.   The self-optimizing vRAN means that network will constantly be in flux.   How can you troubleshoot a network that is always changing?   Due to size of investment, it only makes sense multiple vendors will be used.   Most mobile operators heavily depend upon their NEPs to provide OSS solutions.

The solution is simple, in fact simplification.   A vendor, technology, and product agnostic OSS solution is a must.   As you increase your tools, the complexity limits functionality.   Low-level optimization and orchestration can be done at the element manager layer.   This increases scale of both layers of the solution.


Projecting how 5G will impact operations

Becoming Geospatial Again

Remember the HP OpenView days of maps? When get prepared for those concepts to return. Like wifi antennas, 5G deploys radios with geospatial design in mind. Geospatial information (Lat/Long) will then drive behavior. GIS Correlation and visualization then becomes a need. Correlation and analytics are vital to reducing the complexity of 5G vRAN networks. External network conditions becomes more indicative. Things like hurricanes, floods, and power outages need to be taken into account. This is very similar the cable industry (HFC) access monitoring requirements. Operations will need help because most legacy tools are inadequate in these areas.

Projecting how 5G will impact operations

Bending but not Breaking with Elasticity

Elastic scaling the network is not a 5G concept. The trouble is we have seen that many, if not most, of the network functions are beats. Components like MMEs and SGWs take hours to spin up and configure. This reduces the value of elasticity in 3GPP networks. 5G replaces many complex functions, like eNodeBs, with smaller control functions. This will enable all the promises of network virtualization. The question becomes does operations have orchestration tools to enable automation. Some NEPs are building those functions into the element managers or VNFMs. Most service assurance tools do not have the capability to handle the network flux called by real-time elasticity. Operations will need to review their tools to make sure they are agile enough.

Projecting how 5G will impact operations

Operations Face Uphill Battle

The industry is beating the drums, 5G is coming. But I do not hear from the industry how operations will consume it. From my experience, nobody knows. Legacy tools are too difficult to share information. They are too tied to a vendor or technology domain. Most tools have difficulty scaling to 100k-5m devices. This forces most customers to silo their monitoring and management. This creates lacking visibility capability with drives quality issues. Most operational processes are ticket or fault-centric. Correlation is lagging behind. There will be too many faults to process. Visualization of the network will be a critical need, but may not be possible. Like winter, 5G is coming, so where is my 700 ft wall?


Shawn Ennis Projecting how 5G will impact operations

About the Author

Serial entrepreneur and operations subject matter expert who likes to help customers and partners achieve solutions that solve critical problems.   Experience in traditional telecom, ITIL enterprise, global manage service providers, and datacenter hosting providers.   Expertise in optical DWDM, MPLS networks, MEF Ethernet, COTS applications, custom applications, SDDC virtualized, and SDN/NFV virtualized infrastructure.  Based out of Dallas, Texas US area and currently working for one of his founded companies – Monolith Software.


IoT Service Assurance Key Concepts

The IoT/IoE generation has been born.   Now countless things are about to be inter-connected.   We all see the hype is non-stop, but there many things are becoming a reality.   AT&T/Maersk closed a deal back to 2015.  This recently became a reality for asset tracking cold shipping containers.   Now, Uber is providing driverless trucks to deliver beer.    While GPS trackers are being used to track the elderly.   These services are being ubiquitous and common.   We are seeing the use cases have variety and are growing in depth.   But we also see that IoT is a very pioneering field.   If IoT managed services are to exist, operations will need to manage them.   The goals here is to start asking key questions.   The hope is through analysis we can provide some answers.   Let’s discuss the key concepts driving the new field of IoT Service Assurance.

Key Perspectives for IoT Service Assurance

For any IoT service, you must understand who uses it and who provides it.    As I explain it, there are three key perspectives for IoT services.    First, you have the network provider.   They provide the network access for the “thing”.   The “network” could mean LTE or Wifi or any other technology.     Network providers see the network quality has the focus.  This is similar to typical mobile providers.    Compare that to IoT services monitored with an application focus.  Its about monitoring the availability and performance of the “things”.  You want to make sure they are working.    Lastly, you may not care about the “things“.   Perhaps you only care about the data from the them.   Performing correlation and understanding the “sum of all parts” would be the key focus.   These perspectives drive your requirements and the value prop.    Through them, you can define quality and success criteria for your IoT services.

Key Requirements of IoT Service Assurance

Before we get to far along, let’s first talk about terminology.   In the world of IoT, what is a device?    We have to ask, is this “thing” a device?    With the world of mobility, the handset is not a devices its an endpoint.    So is the pallet being monitored in the cold shipping container a device or an endpoint?  Like the perspectives that drive your requirements, we should agree on terminology.   Let’s talk some use cases to better understand typical requirements.

Cold Storage Tracking IoT Service Assurance

Smart Cold Storage

In the Maersk use case, let’s say the initial roll-out listed as 250k sensors on pallets.   These sensors, at regular intervals, report data in via wireless burst communications.   The data includes KPIs that drive visibility and business intelligence.   Some common examples I have found are: temperature, battery life, and vibration rate.    Other environmental KPIs required can exist: light levels, humidity, and weight.   As we have discussed, location information with signal strength could be useful.   We can track in real-time to provide trend and predication.   One would think it would be best to know a failure before putting the container on the boat.

Bottom line is would have about around 25 KPIs per poll interval.  Let’s do some math for performance data.  Estimate 250k sensors * 25 kpis * 4 (15 min polls, 4/hour) * 24 (hours/day) = 600 million data points per day.   If you were to use a standard database storage (say mysql) you would require 200GB per day.   Is keeping the sensor data worth $300/month per month of data on AWS EC2?   Storage is so inexpensive, real-time monitoring of sensor data becomes realistic.

Now faults are different.  Some could include failed reconnects and emergency button pushed scenarios.    These faults could provide opportunities. Shipping personnel can fix the container before the temperature gets too warm.    Faults could provide an opportunity to save valuable merchandise from spoilage.   Together this information combines to provide detailed real-time IoT Service Assurance views.

Driver-less Trucks for IoT Service Assurance

Driverless Trucks Use Case

Let’s look at another use case: Uber with driverless trucks.   The Wired article does not include how many cars, so let’s look at UPS.   UPS has >100k deliver trucks.    Imagine if these logistics were 100% automated. This would create a tons of “things” on the network.  The network, controller, and data would work together to provide a quality IoT service.

First, let’s look at performance data.  The KPIs should be like the Maersk example.    Speed, direction, location, and range would be valuable real-time data.    Service KQIs like ETA and number of stops remaining would be drive efficiencies.  Let’s do the same math as the Maersk example. Say 100k trucks * 50 kpis * 4 (15 min polls, 4/hour) * 24 (hours/day) = 480 million data points per day.  So $240/day per day on AWS.    This shows that storage and requirements are practical for driverless logistics.

Now some faults would include vital real-time activity.   Perhaps an ‘out-of-gas’ event or network errors.    Getting real-time alerts on crash would definitely be useful.   So fault management would be a necessity in this use case.   Again, there are plenty of reasons to create and leverage real-time alerts.

Smart Home for IoT Service Assurance

Another use case would be smart home monitoring, like Google Nest or Ecobee.   These OTT IoT providers track and monitor things like temperature and humidity.   There is no fault data and no analytics.   The amount of homes monitored by Nest or Ecobee is not readily available on the internet.   According to Dallas News, there are 8 million thermostats sold yearly.   According to Fast Company, Ecobee has 24% marketshare, so 2 million homes per year.   Ecobee has been in business for more than 5 years, so assume they have 10 million active thermostats.  Doing some math, we have 10M homes, 10 kpis * 4 (15 min polls, 4/hour) * 24 (hours/day) = 10 billion data points per day.  So that would be around $4800/day per day on AWS.

IoT Service Assurance is Practical

What is interesting about these use case are their practicality.  Scalability is not a problem with modern solutions. All three cases show that from any perspective. Real-time IoT service assurance is achievable.   I am amazed how achievable monitoring can be for complex and IoT services.  Now you must asked the questions “why” and “how”.   To answer these questions, you must understand how flexible your tools are. What value can you get from them.

Understanding Flexibility of IoT Service Assurance

Let’s discuss flexibility.    First, how difficult is collecting this data?    So let’s focus this in the world of open APIs. The expectation is these messages would come through a load balanced REST application server.   I can image that 600 million hits per day is 2.7k hits/sec.    This is well within apache and load balancer tolerances.   As long as the messaging follows open API concepts collection should be practical. So from a flexibility, assuming you embrace open APIs, this is practical as well.

Understanding the Value of IoT Service Assurance

Its a fact, real-time is a key need in IoT Service Assurance.   If whatever you want to track can wait 24/48 hours before you need to know it, you can achieve it with a reporting tool.   If all you need is to store the data and slap a dashboard/reporting engine on top, then this becomes easy.   Start with open source databases like mariaDB are low cost and widely available. Next, add a COTS dashboards and reporting tools like Tableau provide a cost-effective solution.   

In contrast, Real-time means you need to know immediately that a cold storage container has failed.   Being able to automate dispatch to find the closest human and text that operator to fix the problem.    Real-time means that you have delivery truck on the side of the road and need to dispatch a tow truck.   Real-time IoT Service Assurance means massive collection, intelligent correlation, and automated remediation.  Now let’s look at the OTT smart home as a use case. The NEST thermostat is not going to call the firehouse when it reaches 150F.    Everything is use case dependent, so you must let your requirements dictate the tool used. 

Lessons Learned for IoT Service Assurance

  • IoT-based managed services are currently available and growing
  • Assuring them properly will require new concepts around scalability and flexibility
  • With IoT, you must always ask how far down is it worth monitoring
  • Most all requirements include some sort of geospatial tracking or correlation
 My advice on IoT Service Assurance
  • As always, follow your researched requirements.   Get what you need first, then worry about your wants.
  • Make sure you have tools with a focus on flexibility, scale, and automation.   This vertical has many fringe use cases and they are growing.
  • IoT unifies network, application, and data management more than any other technology.   Having a holistic approach can provide a multiplying and accelerating affect.

About the Author

Shawn Ennis IoT Service Assurance

About the Author

Serial entrepreneur and operations subject matter expert who likes to help customers and partners achieve solutions that solve critical problems.   Experience in traditional telecom, ITIL enterprise, global manage service providers, and datacenter hosting providers.   Expertise in optical DWDM, MPLS networks, MEF Ethernet, COTS applications, custom applications, SDDC virtualized, and SDN/NFV virtualized infrastructure.  Based out of Dallas, Texas US area and currently working for one of his founded companies – Monolith Software.

Predicting the IoT World

What are we going to do in the IoT world?

My typical response to service providers is, “well, that was last week…”    All kidding aside, we live in the connected generation.   Network access is the new oxygen.   The price to be paid is complexity and scale.   A good reference for what IoT use cases exist is this bemyapp article about Ten B2B use cases for IoT.

Common Threads

Its best to categorize them into three buckets.    Environmental monitoring of smart meters to reduce human interaction requirements.   Tracking logistics through RFID is another common trend with IoT communities.   The most common is client monitoring.    With mobility, handset tracking and trending is common in CEM.   When considering an access network its monitoring the cable modems for millions of customers.  Which ever category your use case may be, the challenges will be similar.    How do you deal with the fact that your network becomes tens of millions of small devices instead of thousands of regular sized devices?   How do you handle that fact that billions of pieces of data need to be processed, but only a fraction would be immediately useful?   How can you break down the network to human understandable segmentations?

The solution is simple

With a single source of truth, you can see the forest through the trees.   While the “things” in IoT are important, how they relay information and perform their work are equally important.    Monitoring holistic allows better understanding of the IoT environment – single point solutions will not address IoT.  Normalizing data enables for higher scale, while maintaining the high reliability.

How to accelerate

Now that the network has been unified into a single source of truth, operations can start simplification of their workload.    First step, become service oriented.   Performance, fault, and topology is too much data – its the services you must rely upon.   How are the doing, what are the problems, how to fix them, and where you need to augment your network.    Next up, correlate everything – you need to look at the 1% of the 1% of the 1% to be successful.  KQIs are necessary, because the trees in the forest are antidotal information – the AFFECT.   Seeing the forest (as the KQI) allows you to become proactive and move quicker, be more decisive because you understand the trends and what is normal.  Its time to stop let the network manage you, and start managing your network.

End Goal is Automation

After unifying your view and simplifying your approach, its time to automate.    The whole point of IoT is massive scale and automation, but if your SA solution cannot integrate openly with the orchestration solution, how will you ever automate resolution & maintenance?   We all must realize, human-based lifecycle management is not possible at IoT scale.   Its time to match the value of your network with the value of managing it.