Modern services consist of interconnected components,e.g., microservices in a service mesh or machine learning functions in a pipeline. These services can scale and run across multiple network nodes on demand. To process incoming traffic, service components have to be instantiated and traffic assigned to these instances, taking capacities, changing demands, and Quality of Service (QoS) requirements into account. This challenge is usually solved with custom approaches designed by experts. While this typically works well for the considered scenario, the models often rely on unrealistic assumptions or on knowledge that is not available in practice (e.g., a priori knowledge). We propose DeepCoord, a novel deep reinforcement learning approach that learns how to best coordinate services and is geared towards realistic assumptions. It interacts with the network and relies on available, possibly delayed monitoring information. Rather than defining a complex model or an algorithm on how to achieve an objective, our model-free approach adapts to various objectives and traffic patterns. An agent is trained offline without expert knowledge and then applied online with minimal overhead. Compared to a state-of-the-art heuristic, DeepCoord significantly improves flow throughput (up to 76%) and overall network utility (more than 2x) on realworld network topologies and traffic traces. It also supports optimizing multiple, possibly competing objectives, learns to respect QoS requirements, generalizes to scenarios with unseen, stochastic traffic, and scales to large real-world networks. For reproducibility and reuse, our code is publicly available.

Currently, the coexistence of multiple users and devices challenges the network's ability to reliably connect them. This article proposes a novel communication architecture that satisfies the requirements of fifth-generation (5G) mobile network applications. In particular, this architecture extends and combines ultra-dense networking (UDN), multi-access edge computing (MEC), and virtual infrastructure manager (VIM) concepts to provide a flexible network of moving radio access (RA) nodes, flying or moving to areas where users and devices struggle for connectivity and data rate. Furthermore, advances in radio communications and non-orthogonal multiple access (NOMA), virtualization technologies and energy-awareness mechanisms are integrated towards a mobile UDN that not only allows RA nodes to follow the user but also enables the virtualized network functions (VNFs) to adapt to user mobility by migrating from one node to another. Performance evaluation shows that the underlying network improves connectivity of users and devices through the flexible deployment of moving RA nodes and the use of NOMA.

Assigning bands of the wireless spectrum as resources to users is a common problem in wireless networks. Typically, frequency bands were assumed to be available in a stable manner. Nevertheless, in recent scenarios where wireless networks may be deployed in unknown environments, spectrum competition is considered, making it uncertain whether a frequency band is available at all or at what quality. To fully exploit such resources with uncertain availability, the multi-armed bandit (MAB) method, a representative online learning technique, has been applied to design spectrum scheduling algorithms. This article surveys such proposals. We describe the following three aspects: how to model spectrum scheduling problems within the MAB framework, what the main thread is following which prevalent algorithms are designed, and how to evaluate algorithm performance and complexity. We also give some promising directions for future research in related fields.

The ongoing softwarization of networks creates a big need for automated testing solutions to ensure service quality. This becomes even more important if agile environments with short time to market and high demands, in terms of service performance and availability, are considered. In this paper, we introduce a novel testing solution for virtualized, microservice-based network functions and services, which we base on TTCN-3, a well known testing language defined by the European standards institute (ETSI). We use TTCN-3 not only for functional testing but also answer the question whether TTCN-3 can be used for performance profiling tasks as well. Finally, we demonstrate the proposed concepts and solutions in a case study using our open-source prototype to test and profile a chained network service.

Softwarized networks are the key enabler for elastic, on-demand service deployments of virtualized network functions. They allow to dynamically steer traffic through the network when new network functions are instantiated, or old ones are terminated. These scenarios become in particular challenging when stateful functions are involved, necessitating state management solutions to migrate state between the functions. The problem with existing solutions is that they typically embrace state migration and flow rerouting jointly, imposing a huge set of requirements on the on-boarded virtualized network functions (VNFs), eg, solution-specific state management interfaces. To change this, we introduce the seamless handover protocol (SHarP). An easy-to-use, loss-less, and order-preserving flow rerouting mechanism that is not fixed to a single state management approach. Using SHarP, VNF vendors are empowered to implement or use the state management solution of their choice. SHarP supports these solutions with additional information when flows are migrated. In this paper, we present SHarP's design, its open source prototype implementation, and show how SHarP significantly reduces the buffer usage at a central (SDN) controller, which is a typical bottleneck in state-of-the-art solutions. Our experiments show that SHarP uses a constant amount of controller buffer, irrespective of the time taken to migrate the VNF state.

In many cyber–physical systems, we encounter the problem of remote state estimation of geo- graphically distributed and remote physical processes. This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes. Information from the different sensors has to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored. For effective estimation at the gateway, the sensors need to be scheduled appropriately, i.e., at each time instant one needs to decide which sensors have network access and which ones do not. To address this scheduling problem, we formulate an associated Markov decision process (MDP). This MDP is then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free. We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waiting-time, among others. Our algorithm is shown to significantly outperform these algorithms for many example scenario

To adapt to continuously changing workloads in networks, components of the running network services may need to be replicated (scaling the network service) and allocated to physical resources (placement) dynamically, also necessitating dynamic re-routing of flows between service components. In this paper, we propose JASPER, a fully automated approach to jointly optimizing scaling, placement, and routing for complex network services, consisting of multiple (virtualized) components. JASPER handles multiple network services that share the same substrate network; services can be dynamically added or removed and dynamic workload changes are handled. Our approach lets service designers specify their services on a high level of abstraction using service templates. JASPER automatically makes scaling, placement and routing decisions, enabling quick reaction to changes. We formalize the problem, analyze its complexity, and develop two algorithms to solve it. Extensive empirical results show the applicability and effectiveness of the proposed approach.

Network function virtualization and software-defined networking allow services consisting of virtual network functions to be designed and implemented with great flexibility by facilitating automatic deployments, migrations, and reconfigurations for services and their components. For extended flexibility, we go beyond seeing services as a fixed chain of functions. We define the service structure in a flexible way that enables changing the order of functions in case the functionality of the service is not influenced by this, and propose a YANG data model for expressing this flexibility. Flexible structures allow the network orchestration system to choose the optimal composition of service components that for example gives the best results for placement of services in the network. When number of flexible services and number of components in each service increase, combinatorial explosion limits the practical use of this flexibility. In this paper, we describe a selection heuristic that gives a Pareto set of the possible compositions of a service as well as possible combinations of different services, with respect to different optimization objectives. Moreover, we present a heuristic algorithm for placement of a combination of services, which aims at placing service components along shortest paths that have enough capacity for accommodating the services. By applying these solutions, we show that allowing flexibility in the service structure is feasible.

A current trend in networking and cloud computing is to provide compute resources at widely distributed sites; this is exemplified by developments such as Network Function Virtualisation. This paves the way for wide-area service deployments with improved service quality: e.g. user-perceived response times can be reduced by offering services at nearby sites. But always assigning users to the nearest site can be a bad decision if this site is already highly utilised. This paper formalises two related decisions of allocating compute resources at different sites and assigning users to them with the goal of minimising the response times while the total number of resources to be allocated is limited – a non-linear capacitated Facility Location Problem with integrated queuing systems. To efficiently handle its non-linearity, we introduce five linear problem linearisations and adapt the currently best heuristic for a similar scenario to our scenario. All six approaches are compared in experiments for solution quality and solving time. Surprisingly, our best optimisation formulation outperforms the heuristic in both time and quality. Additionally, we evaluate the influence of distributions of available compute resources in the network on the response time: The time was halved for some configurations. The presented formulation techniques for our problem linearisations are applicable to a broader optimisation domain.

The Service Programming and Orchestration for Virtualised Software Networks (SONATA) project targets both the flexible programmability of software networks and the optimisation of their deployments by means of integrating Development and Operations in order to accelerate industry adoption of software networks and reduce time-to-market for networked services. SONATA supports network function chaining and orchestration, making service platforms modular and easier to customise to the needs of different service providers, and introduces a specialised Development and Operations model for supporting developers.

Financial benefits are an important factor when cloud infrastructure is considered to meet processing demand. The dynamics of on-demand pricing and service usage are investigated in a two-stage game model for a monopoly Infrastructure-as-a-Service (IaaS) market. The possibility of hybrid clouds (public clouds plus own infrastructure) turns out to be essential in order that not only the provider but also the clients have significant benefits from on-demand services. Even if the client meets all demand in the public cloud, the threat of building a hybrid cloud keeps the instance price low. This is not the case when reserved instances are offered as well. Parameters like load profiles and economies of scale have a huge effect on likely future pricing and on a cost-optimal split-up of client demand between either a client’s own data center and a public cloud service or between reserved and on-demand cloud instances.