Flexible VM Provisioning for Time-Sensitive Applications with Multiple Execution Options: Computer Science Final Year Project Singapore

Flexible VM Provisioning for Time-Sensitive Applications with Multiple Execution Options

I. INTRODUCTION

Cloud computing platforms are rapidly emerging as the preferred option for hosting applications in many business contexts [7]. It enables on-demand and flexible provisioning of a shared pool of hardware resources (e.g., CPU, memory, disks and networks) to user applications. There is a myriad of technologies and services that are required for the effective operation of cloud computing systems.

One of the most crucial services for resource efficiency of cloud computing is the effective resource provisioning strategy. Undoubtedly resource provisioning (i.e., the selection and deployment of VMs of user jobs) is an important and challenging problem in cloud computing environments and has been intensively studied in prior research [3], [2], [31], [21], [6]. Although prior studies have investigated the VM provisioning problem for time-sensitive jobs (which have deadlines) on resource-constrained clouds, they usually assumed that a user request (job) has only one execution option (denoted by its

execution time with a certain resource demand). However, modern applications, such as high-performance computing applications and machine-learning applications, are usually implemented with parallel frameworks, such as Pthreads, OpenMP, MPI and Tensorflow [25], [9], [12], [1]. These frameworks allow modern applications to executed with various numbers of threads on servers with different resources.

Therefore, when submitting a request (job) to a cloud system, a user may have several options on how the application can be executed. Here, each option can be specified by a requirement of resources in terms of VM sizes and VM counts, as well as a unique estimated execution time of the application based on the resource demand

When there are a range of execution options for executing an application, it becomes difficult and too rigid for the user to choose only one (or the best) execution option. More importantly, our previous experience in cloud VM provisioning has shown that one major cause of low system utilization is this rigid resource demand model, which limits the provisioner’s ability to fully utilize the un-allocated resource fragments.

Specifically, if a job is specified with only one execution option that demands more resources than any un-allocated resource fragments, such an option will prevent the job from being issued for execution, even if the job’s implementation may allow it to use fewer numbers of threads and VMs. As shown in Section III, having multiple execution options (MEOs) for each job provides opportunities for more flexible VM provisioning.

Note that, simply adding more execution options does not guarantee improved system utilization and benefits. In fact, it may even lead to worse performance if improper options are chosen as most parallel applications do not have perfect linear speedups and normally experience lower resource efficiency for systems with more resources [26].

For instance, if a highresource demand execution option is chosen for a job, it may block more resources for an extended period than low-resource demand option due to lower resource efficiency, which could prevent the resources from being allocated to other jobs and in turn lead to more jobs missing deadlines and lower system benefits.

Consequently, to realize the full potential of multiple execution options of jobs, VM provisioning algorithms must be carefully designed to balance system utilization, resource efficiency, input workloads, job deadlines and benefits.

visioning schemes named MEO-Greedy and MEO-Adaptive. Here, both schemes consider the MEOs of requests and prioritize them based on current system load, resource efficiency of execution options, deadlines and job benefits (bids). With the priorities, requests are provisioned with resources by either selecting the more efficient execution options to increase system utilization, or adapting (scaling up) the execution options gradually to maximize resource utilization with most efficient options.

The proposed schemes were evaluated using a simulator for various workloads (e.g., overload and spiky scenarios) with requests of both benchmark and synthetic applications. The results show that our MEO-aware schemes outperform the state-of-the-art schemes that consider only a single execution option of jobs by serving up to 38% more jobs and achieving up to 27% more benefits

II. CLOSELY RELATED WORKS

The problem of VM provisioning in cloud systems has been investigated from different points of view and many research studies have been conducted on resource allocation mechanisms [4], [34]. For instance, the provisioning techniques have been investigated to improve the performance of the user applications [20], [21], [14], to efficiently use cloud resources [35], [16], to minimize user cost, and maximize the revenue for cloud service providers [36], [11], [27], [28], to deliver services to the cloud users even in presence of failures [18], [17], to improve QoS parameters [30], [32], or to reduce power consumption [19], [15]. However, only very limited research has focused on time-sensitive applications with deadlines

III. PRELIMINARIES AND MOTIVATING EXAMPLE

A. Cloud and VM Models

We consider a cloud system that consists of M heterogeneous computing nodes Π = {N1,…, NM }. Each node has R types of resources Γ = {R1,…, RR} (such as CPU cores, memory and network bandwidth). The capacity vector of node Np is denoted as Cp = (c1 p,…,cR p ), where cr p represents the total capacity of resource Rr on node Np (e.g., number of CPU cores).

With heterogeneous nodes being considered, the capacity of one resource in different nodes can be different. The computing resources in the cloud can be accessed by cloud users in the form of virtual machines (VMs).

In this work, we consider V types of virtual machines {V1,…, VV } that have different resource requirements. For the VM type Vk, its required resources can be denoted by a demand vector Wk = (w1 k,…,wR k ), where wrk represents the required capacity (or amount) of resource Rr.

B. Requests with Multiple Execution Options (MEOs) To run a time-sensitive application, a cloud user needs to submit a request to the cloud system, which can be represented as a tuple θi = (ai, di, bi, Ei). Here, ai denotes the request’s arrival time, di defines the deadline by which the application needs to complete its execution, and bi represents the bid (or benefit) that the system can achieve when the application completes its execution in time.

Since an application may run with different number of threads on one or more VMs with different number of cores (and other resources), a cloud user can specify multiple execution options (MEOs) for running the application, which form the execution vector Ei = !!vi,1, mi,1, ti,1″,!vi,2, mi,2, ti,2″,…,!vi,j , mi,j , ti,j “”.

Here, the kth option is denoted as !vi,k, mi,k, ti,k”, where vi,k (∈ [1, V ]) and mi,k represent the type and number of VMs needed to run the application, and ti,k denotes its (estimated) worst-case execution time on the deployed VMs

C. A Motivating Example

For simplicity, we consider a cloud system that has only two nodes N1 and N2 with 4 CPU cores, 8GB memory and 16 CPU cores, 32GB memory respectively. That is, C1 = (4, 8) and C2 = (16, 32). Three types of virtual machines V1, V2 and V3 are considered with their demand vectors as W1 = (2, 4), W2 = (4, 8) and W3 = (8, 16) respectively.

Here, each request has two or more execution options. Since most applications do not scale perfectly, for an option with more cores (and other resources), the execution time of the application with more threads does not decreases linearly, which actually leads to increased workload. For the requests in Table I, their least-demanding (most-demanding) options result in around 50%, 90% and 120% (160%, 180% and 280%) system loads for the three intervals, respectively.

IV. FLEXIBLE MEO-AWARE VM PROVISIONING

A. Overview of MEO-TSRA

Due to the NP-hard nature of the task scheduling problem in general [13], heuristics are usually adopted in resource provisioning approaches in clouds. In this paper, we present two MEO-aware time-sensitive resource allocation (MEO-TSRA) heuristic schemes. These two heuristic schemes balance the needs of improving system utilization and resource efficiency to increase job deadline satisfactions and achieved benefits.

B. Prioritization of User Requests

The goal of request prioritization is to determine the requests that should be executed earlier than the other requests. Intuitively, requests with higher benefits and higher urgency should be executed earlier. Furthermore, requests with lower

C. MEO-Greedy

MEO-Greedy is our first resource allocation heuristic scheme. Algorithm 2 gives the pseudo-code of this scheme. MEO-Greedy takes the prioritized jobs and available resources as input parameters. It then iterates over the requests from the highest priority to lowest priority (line 3 and 4) to allocate resources for them. For request θi, MEO-Greedy scans over its execution options one by one and greedily pick the first option that is feasible to be allocated (line 6-16).

When scanning the options, MEO-Greedy may either process the options from the first (slowest and least-demanding) option to the last (fastest and most-demanding) option (which is called MEOGreedy-First), or it may go from the last to the first (which is called MEO-Greedy-Last).

Intuitively, MEO-Greedy-First picks the option with the highest resource efficiency, while MEO-Greedy-Last picks the option with the highest chance of increasing overall system utilization. If the current option E is feasible (line 9), it will be allocated and mapped to a node to execute (line 10-13).

We use a Euclidean Distance (ED) mapping strategy presented in our recent work [3]. If none of the options of θi can be allocated due to resource limitation, θi will be added to wait queue (line 19). If all of θi’s options miss its deadline, θi misses deadline and is discarded (line 21).

D. MEO-Adaptive

MEO-Adaptive is our second resource allocation scheme, the pseudo-code of which is given in Algorithm 3. MEOAdaptive first allocates resources to the requests using their first (slowest) options (line 2). This allocation step is similar to the MEO-Greedy algorithm except it always uses the first option. Due to space limitation, the exact algorithm of this first-option allocation is not included in the paper. If there are resources available after the above allocation, MEO-Adaptive tries to scale-up the requests until most of the resources are allocated (line 3). To scale-up, MEO-Adaptive examines the next (faster and more demanding) options of all requests to pick the request with the most scalable next-option (line 10-11) and scale-up this request with this option (line 13- 14). MEO-Adaptive repeats this scale-up operation until there is not enough resource left to scale-up any requests (line 9). Intuitively, MEO-Adaptive aims at maximizing resource utilization with the most efficient options

V. EVALUATIONS AND DISCUSSIONS

We have evaluated the performance of the proposed MEOaware VM provisioning schemes through extensive simulations. In addition to the proposed MEO-Greedy-First, MEOGreedy-Last and MEO-Adaptive schemes, for comparison, we also evaluated the TSRA scheme in our prior work [2] by considering only the first and last execution options of requests, which are denoted as TSRA-First and TSRA-Last, respectively. In what follows, we first present the settings of the simulations.

A. Simulation Settings

Although the proposed VM provisioning schemes can handle multiple types of resources, we have focused on only two types of resources (i.e., CPU cores and memory) in the evaluations. We considered a cloud system with 16 heterogeneous computing nodes, where two nodes have the capacity of (16c, 64GB), two nodes of (8c, 32GB), four nodes of (4c, 8GB) and eight nodes of (2c, 4GB), which represent a mixture of typical computing nodes.

B. Performance for Requests of NAS Benchmark Applications For the applications in the NAS Benchmarks suite, we have run them in both OMP and MPI versions on different numbers and types of VMs in the OpenStack environment installed on a cluster of 10 computing nodes. The detailed execution times for these applications running on different VMs can be found in the technical report of [3]. Based on the collected trace data regarding the applications’ execution times on different VMs, the user requests to utilize the computing resources of the considered cloud system can be generated.

C. Performance for Synthetic Requests

To eliminate the impacts of irregular resource demands in execution options of benchmark applications, synthetic requests with more regular execution options are considered. More specifically, for each synthetic request, we first arbitrarily pick a VM from Table II with the execution time being randomly chosen between 5 to 200 minutes as its first execution option. Then three (3) more execution options are generated by increasing the resource demands gradually. For two consecutive options Ei,k and Ei,k+1, the resource demand is usually increased by a ratio of x, where x is randomly picked between 1 and 3. The execution time reduces by a factor between 1 and x to reflect the non-linear scalability.

Figure 4 shows the performance of all schemes for synthetic requests with different loads. In general, the results show the similar trends for the schemes when system load changes. However, as the execution options of requests become more regular in resource demands, all MEO-aware schemes outperform TSRA, where MEO-Adaptive performs the best as it can adaptively choose the most resource-efficient execution options for requests and improve overall system utilization.

VI. CONCLUSIONS

In this work, we studied the VM provisioning problem for time-sensitive requests with multiple execution options (MEO) on resource-constrained clouds. We demonstrated that MEOs allow more flexible resource allocation which can considerably increase system utilization, deadline satisfaction and overall system benefit.

To realize the full potential of MEO, we present two MEO-aware VM provisioning schemes, MEOGreedy and MEO-Adaptive, that are carefully designed to balance overall system utilization, resource efficiency, request deadlines and benefits.

We evaluated the proposed MEOaware schemes on various workloads with both benchmark requests and synthetic requests. The results show that our MEO-aware algorithms outperform the state-of-the-art single execution-option schemes by serving up to 38% more requests and achieving up to 27% more benefits.

The post Flexible VM Provisioning for Time-Sensitive Applications with Multiple Execution Options: Computer Science Final Year Project Singapore appeared first on Singapore Assignment Help.