{"template":"sigconf","title":"Size-Biased Admission in Print-Gated GPU Clusters","authors":[{"name":"Mara Venn","affiliations":[1]}],"affiliations":["Calden Institute for Computing, Somerville, MA, USA"],"abstract":"A print-gated scheduler issues a placement request only after one allocation sheet per requested GPU has reached the printer's output tray. If a jam invalidates the partly printed ticket, wide gangs need more attempts and rejoin the spool more often. We derive the offered load of this admission stage and evaluate it by discrete-event replay on a 64-GPU cluster model. At a 7% per-feed jam probability, eight-GPU tickets discard 2.47 ejected sheets per admission. Compared with an otherwise identical jam-free printer, jams reduce the scheduler's 95th-percentile placement wait from 110 to 93 s while increasing the client's 95th-percentile wait from 121 to 561 s. GPU utilization differs by 0.35 percentage points. A 15 min interruption leaves the placement queue empty and all GPUs idle while an average of 174 requests wait in the spool. Fitting two accelerator records per sheet retains the reprint rule and lowers the client tail to 135 s. The client clock includes the delay at the print spool.","keywords":["GPU clusters","gang scheduling","admission control","queueing delay","print spooling"],"body":"# Introduction {#sec:intro}\n\nA GPU scheduler normally timestamps a job when it enters the placement queue. In an installation that requires a printed allocation ticket, this timestamp can be substantially later than submission. Consider a shared cluster that keeps one tear-off sheet per requested accelerator for its allocation records. The spooler issues the placement request after the last sheet reaches the output tray. A paper jam delays the request, but the delay does not appear in the placement queue because the request has yet to be issued.\n\nThe distinction matters for gang jobs. An eight-GPU job needs a node that can accommodate the whole gang [@gu2019; @jeon2019]. Under a one-sheet-per-accelerator rule it also needs eight consecutive successful feeds. A jam invalidates the partly printed ticket. When cleared, the printer moves the interrupted request to the back of the spool, where it can be overtaken by requests with fewer pages. Thus a printer can impose a size preference without reading a job's priority, runtime, or placement constraints.\n\nWe analyze this form of admission under three questions. How much of the preference comes from losing the prefix of a ticket, as opposed to the time needed to clear a jam? What does a queue monitor report when admission becomes the bottleneck? Can a change to the print layout relieve the bottleneck while preserving one physical record for each device? We answer these questions with a service-demand model and a discrete-event replay. An interrupted feed changes the order in which the scheduler sees jobs, and separating the spool from the scheduler changes the measured queueing tail even when the overall fraction of busy GPUs changes little.\n\n# Print-Gated Admission {#sec:design}\n\n## Grant publication\n\nA submission carries a request ID and a gang width $g$. The print ticket consists of $g$ sheets, each bearing the request ID, the print attempt number, and a distinct slot ordinal. The IDs name requested slots. Physical device IDs are assigned later, when the cluster has room for the gang. The subscriber at the scheduler receives a grant only after ejection of every sheet in the attempt. Until then, the spool retains the request and no device is held for it.\n\nThe Internet Printing Protocol defines per-job completion counters for media sheets [@ipp]. Our gate assumes an ejection record with a request ID and sequence number. For admission, a last page counter by itself is insufficient. On restart, a complete set of slot ordinals from an active attempt is required. A partially ejected set is void, even if it remains in the tray. The spool retains the logical ID across retries. The scheduler consumes the grant at most once per logical ID, so repeating notification after ejection cannot admit a second gang. The evaluation treats spool state and admission publication as reliable and models neither crashes of the spool host nor duplicate messages.\n\nIf a feed jams, the engine is blocked during clearance and processes no other request. At the end of clearance, it appends the interrupted job at the tail with a fresh attempt number. Already admitted jobs continue to execute. When sheets are unavailable, the engine instead waits for stock and resumes the current attempt. The scheduler can therefore distinguish an empty placement queue from an empty system only if it can observe the spool.\n\n## Printer work per request\n\nLet $m$ be the number of sheets required for one ticket, $p$ the independent probability that a feed jams, and $q=1-p$. For the default layout $m=g$. A complete first attempt has probability $q^m$. With eventual retry, the number of attempts is geometric with mean $q^{-m}$. This counts attempts per request regardless of intervening requests. The wall time between attempts depends on where the request rejoins the queue.\n\nPrinting a valid ticket requires a run of $m$ non-jammed feeds for that request. Let $N_m$ count the feeds expended across all its attempts, including the feeds ending in a jam. Conditioning on the next feed at each prefix length, with a jam resetting the prefix to zero, gives\n$$\n\\mathbb{E}[N_m] = \\frac{q^{-m}-1}{p}.\n\\label{eq:feeds}\n$$\nThe limit at $p=0$ is $m$. Each feed takes $s$ seconds, and each jam adds a clearance of $c$ seconds to the engine's occupied time. The expected occupied time per ticket is $(s+pc)\\mathbb{E}[N_m]$. The expected number of ejected sheets discarded before a valid ticket is $q\\mathbb{E}[N_m]-m$. A feed that jams does not count as an ejected sheet.\n\nFor job arrival rate $\\lambda$ and width probabilities $\\pi_g$, the offered printer load is\n$$\n\\rho_P = \\lambda(s+pc)\\sum_g \\pi_g\\mathbb{E}[N_{m(g)}].\n\\label{eq:load}\n$$\nThis expression accounts for retries even when the cluster has spare devices. If $\\rho_P>1$, a single printer cannot issue grants at the offered rate in the long run. The scheduler does not backpressure the printer, so the threshold is independent of the number of free devices. This calculation predicts the location of the printer bottleneck. It does not predict the distribution of wait times at either queue.\n\n## Recovery and layout controls\n\nWe compare the default policy, **Restart**, with five controls. **Digital** publishes immediately and incurs no print time. **Jam-free** runs the default one-sheet-per-GPU layout with $p=0$. This controls for the time required to print a complete ticket. **Head restart** discards the same prefix as Restart, but holds the head of the spool until it completes a ticket. It has the same expected service demand as Restart and admits jobs in submission order.\n\n**Resume** also holds the head of the spool, preserves each ejected sheet, and retries only the jammed feed, with the same clearance time. Its expected feed count is $m/q$. This policy requires a reliable record of which ordinals were ejected before clearance.\n\n**Two-up** prints at most two adjacent slot records per page. The two records carry their own ordinals and can be separated after ejection. Here $m=\\lceil g/2\\rceil$, and a jam still discards the whole prefix and moves the job to the tail. The engine's ejection time remains $s$ because both records are printed on the same face. The two-up control assumes that the custodian accepts cut sections as individual records. Sites requiring a full page for a slot would have to use a different policy.\n\n# Evaluation Method {#sec:method}\n\n## Workload and replay\n\nWe model eight homogeneous servers with eight GPUs each. A job occupies all requested accelerators on one node for its entire runtime. At grant issuance and job completion, the scheduler scans the admitted queue in order. It places each job on the node with the fewest free GPUs that can still hold the gang. A job that cannot fit remains in the queue while subsequent jobs can start. The scheduler makes no reservations for skipped jobs and does not use predicted runtimes. This placement rule and all runtimes are held fixed across policies.\n\nLarge GPU traces document gang and locality constraints [@jeon2019]. We use a synthetic workload rather than estimate print demand from those traces, which contain no such field. Widths are independent draws of 1, 2, 4, or 8 GPUs with probabilities 0.58, 0.22, 0.12, and 0.08. The corresponding mean runtimes are 90, 120, 160, and 240 s. Within each class we draw lognormal runtimes with a log-scale standard deviation of 0.65. The mean job uses 335.4 GPU-s. For nominal GPU load $\\rho_G$, we set $\\lambda=64\\rho_G/335.4$ jobs/s. Arrivals follow a Poisson process and continue during a jam, so the simulated client does not reduce load in response to delayed grants [@schroeder2006].\n\nThe reference configuration uses 1.5 s per feed, 12 s of clearance per jam, $p=0.07$, and $\\rho_G=0.82$. These are controlled settings, independent of any particular printer model. We sweep jam probability, clearance time, and nominal GPU load independently. An additional replay interrupts feed service for 15 min at hour 4. A feed that would overlap this interruption is deferred until service returns. Spool state and already ejected sheets survive this interruption. In each control and outage pair, we use the same workload and the same stream of jam draws by feed order.\n\n## Measurements\n\nWe run 32 independent traces per setting, with workload seeds 101 through 132 and a separate jam stream seeded with the workload seed plus 10,000. Each trace starts empty. We measure jobs submitted from hour 2 up to hour 8, keep generating arrivals for one further hour, and then drain both queues. All 108,104 submissions in this window at the reference load are included in each policy's latency estimates. Of these, 8,714 request eight GPUs. At settings for which printer demand exceeds capacity, this procedure yields a finite-replay delay, and comparisons at those settings depend on run duration.\n\nThe client clock starts at submission and the placement clock starts at grant publication. Both stop when the gang starts. We report the mean across traces of the within-trace $p_{95}$ computed with linear interpolation. A printed job's spool time is the difference between its two wait measurements. Quantiles of these components are computed separately. We compute paired 95% percentile bootstrap intervals from 5,000 resamples of trace-level differences. A submission counts as overtaken if at least one later submission receives its grant before it does. We determine this using grant order, regardless of the order in which the jobs start.\n\nGPU utilization is busy GPU time divided by capacity during the six-hour measurement window, including work that started before the window. The outage replay additionally samples both queue lengths and busy GPU counts every 30 s. A ticket currently being printed or cleared counts toward the spool length.\n\n# Results {#sec:results}\n\n## Paper loss scales with gang width\n\n@tbl:tickets compares the model and replay at the reference settings. The first-attempt success probability falls from 0.93 at width one to 0.56 at width eight. The observed eight-GPU request uses 1.79 attempts and discards 2.47 ejected sheets on average. The waste is concentrated in the largest gangs. They account for 8.1% of submissions and about 73% of all ejected sheets later voided. A failed one-sheet attempt has no prefix to discard.\n\nTable: Analytical and replayed ticket costs. $P_1$ is the analytical probability of completing the first attempt. Attempts and discarded sheets are replay means by width. {#tbl:tickets}\n\n| GPUs | $P_1$ | Attempts | Discarded sheets |\n|---:|---:|---:|---:|\n| 1 | .930 | 1.073 | 0.000 |\n| 2 | .865 | 1.153 | 0.072 |\n| 4 | .748 | 1.347 | 0.482 |\n| 8 | .560 | 1.792 | 2.465 |\n\nThe model puts reference printer load at 94.9%. Setting clearance to zero keeps the width-dependent retry rule and leaves the spool charged for failed feeds. At this setting the overall client $p_{95}$ is 128.4 s. Increasing clearance to 4, 8, and 12 s raises it to 147.4, 208.0, and 560.6 s, respectively. Clearance time is felt by every job behind a jam, including jobs whose own ticket would print successfully on the first attempt.\n\nTail re-entry changes who pays this delay. Under Restart, a later submission receives its grant first for 6.8% of one-GPU jobs and 43.7% of eight-GPU jobs, averaging the proportions across traces. Head restart eliminates this overtaking while leaving the same sheets subject to loss. Returning the failed ticket to the tail adds 455.0 s to the wide-gang client $p_{95}$ (95% CI 290.3 to 650.6 s). The corresponding increase for all widths is 54.3 s (95% CI -6.7 to 125.5 s). Most submissions use one or two GPUs, so the aggregate delay obscures the penalty at width eight.\n\n## Jams improve the reported placement tail\n\nAt the reference point, Restart lowers the placement $p_{95}$ by 17.0 s against Jam-free (95% CI 2.0 to 33.4 s lower). The client $p_{95}$ rises by 439.4 s (95% CI 332.9 to 565.5 s higher). For eight-GPU gangs, the two effects are larger: the placement tail falls by 69.8 s (95% CI 22.4 to 120.3 s lower) and the client tail rises by 674.8 s (95% CI 480.9 to 903.2 s higher). @tbl:policy gives the absolute waits and utilization. The utilization difference between Restart and Jam-free is -0.35 percentage points, with a paired 95% interval of -0.78 to 0.02 points.\n\nTable: Mean per-trace 95th-percentile waits, in seconds. P measures from placement-queue entry, C from client submission. Util. is cluster-wide GPU utilization (%). {#tbl:policy}\n\n| Policy | All P | All C | 8-GPU P | 8-GPU C | Util. |\n|:---------|----------------:|-------------:|----------------:|-------------:|---------------------:|\n| Digital | 109.9 | 109.9 | 427.5 | 427.5 | 82.4 |\n| Jam-free | 110.3 | 121.2 | 418.1 | 434.5 | 82.4 |\n| Restart | 93.3 | 560.6 | 348.3 | 1,109.3 | 82.0 |\n| Head restart | 102.8 | 506.3 | 327.8 | 654.2 | 82.3 |\n| Resume | 107.4 | 202.2 | 388.3 | 466.1 | 82.4 |\n| Two-up | 116.2 | 134.7 | 417.3 | 438.8 | 82.4 |\n\n@fig:sweep traces the effect as faults become more frequent. At $p=0.08$, modeled printer load exceeds 100% and the overall client $p_{95}$ reaches 1,758.5 s in the finite replay. The placement tail is 69.8 s, the lowest in the sweep. At $p=0.07$, the same split appears at high GPU demand. Moving from $\\rho_G=0.62$ to 0.90 raises Restart's client tail from 107.4 to 2,073.5 s and crosses the printer's capacity threshold. At load 0.90, bypassing the printer yields 296.5 s on both clocks, more than twice the placement wait recorded under Restart (146.5 s).\n\n![Jam-probability sweep under Restart at 82% nominal GPU load. Each point averages 32 within-trace 95th percentiles. At 8% jams the spool is overloaded and the client estimate corresponds to a six-hour submission window.](sweep){#fig:sweep}\n\n## Changing the page layout\n\nResume lowers the reference client tail from 560.6 to 202.2 s. The reduction is 358.3 s (95% CI 258.5 to 480.6 s). By retaining every ejected sheet, the policy reduces offered printer load to 84.3%. The client tail under Head restart is 506.3 s with the same first-in, first-out grant order as Resume. Holding the ticket at the head is insufficient on its own when a jam still wastes the whole prefix.\n\nTwo-up retains tail re-entry, reduces eight-GPU tickets to four sheets, and leaves the one-GPU ticket unchanged. At $p=0.07$, it raises predicted first-attempt success at width eight from 0.560 to 0.748; the replayed mean number of attempts falls from 1.79 to 1.33. Offered printer load drops to 55.4%, and the overall client tail falls to 134.7 s. Placement wait rises to 116.2 s. The cluster receives more of the waiting work directly. A dashboard that only shows placement delay would favor Restart over both recovery options.\n\n## A paper-out incident\n\n@fig:outage interrupts printing with both queues already active. Five minutes after interruption, the placement backlog has emptied in every treated trace. The spool has grown from a mean of 31.7 to 80.2 requests, while the mean busy GPU count has fallen from 53.1 to 11.8. By minute 15, all 64 GPUs are idle in every treated trace and the spool holds a mean of 174.2 requests. The control has 54.0 busy GPUs and a mean spool length of 31.2 at the same time.\n\nTwenty minutes after replenishment the treated cluster is using 54.2 GPUs on average, against 52.4 without an interruption, but the mean spool still holds 150.3 requests against 36.7. At the reference workload the long-run printer service budget is 9.89 tickets/min against 9.39 incoming requests/min. That margin is too narrow for the return of busy GPUs to imply that the spool has recovered. A single status panel tracking only placement and utilization shows the beginning and end of a capacity incident on a different schedule from the one experienced by clients.\n\n![Response to a 15 min interruption in feed service. Samples at 30 s intervals are means across 32 paired replays. Panel (a) counts jobs waiting at the printer, including the active ticket, and jobs waiting for placement. Dashed lines show uninterrupted controls. Vertical rules delimit the interruption.](outage){#fig:outage span=full}\n\n# Related Work {#sec:related}\n\nTiresias uses attained service and placement decisions to reduce job completion time in GPU clusters [@gu2019]. Gandiva time-slices and migrates jobs to improve assignment fit [@xiao2018], and Pollux changes both job configurations and cluster allocations to improve goodput [@qiao2021]. Themis explicitly considers how shared cluster service affects finish times [@mahajan2020]. In all these cases, the policy is evaluated over jobs known to the scheduler. Here the policy under study decides when a job becomes known, without modifying the scheduler's placement or time-sharing rules.\n\nSystems work on latency tails gives reasons to measure at the caller [@dean2013]. A placement wait is meaningful for diagnosing slow placement after admission. It does not account for the backlog at an upstream service. The print gate makes this distinction observable because the arrival order at the placement queue is itself an outcome of per-sheet failures. The two clocks are well defined even when both queues eventually drain.\n\n# Discussion {#sec:discussion}\n\nThe paper handling rule assigns a scope to the failure. Restart invalidates a whole gang ticket after one bad feed, whereas Resume limits redo to one sheet. This is separate from the question of whether a retry should take its old place in the spool. A production gate would need to log the request ID at submission, preserve that ID across attempts, and reconcile it against the grant log. Recording printer uptime or page throughput alone would not reveal which job widths are still waiting. Counting sheets instead of job IDs would further overweight wide tickets and repeated attempts.\n\nThis study uses one print engine, independent faults, and synthetic fixed-width jobs. A damaged roller can concentrate faults in time, and a shared office printer may also receive unrelated jobs. Neither effect appears in the model. We assume successful ejection is observed exactly and that restoration after a paper-out preserves the active ticket. We have no hardware measurements of clearance time, and the load threshold depends on that parameter. Our intervals describe variability across synthetic traces. They do not quantify uncertainty about print hardware or workloads in the field. An elastic scheduler that can resize jobs before grant publication would also change the distribution of ticket lengths. Resizing jobs after admission would leave the initial paper demand intact.\n\nThe cleanest operational measure is time from the first submission carrying a logical request ID. It includes tickets waiting on clearance and requests that later require additional attempts. In the evaluated design a job becomes runnable when a complete ticket arrives, but it becomes a waiting job as soon as the spool accepts it. The two events should have separate timestamps even when both records are eventually kept in the same file.\n","bibliography":"@article{dean2013,\n  author = {Jeffrey Dean and Luiz Andr{\\'e} Barroso},\n  title = {The Tail at Scale},\n  journal = {Communications of the ACM},\n  year = {2013},\n  volume = {56},\n  number = {2},\n  pages = {74--80},\n  doi = {10.1145/2408776.2408794}\n}\n\n@inproceedings{gu2019,\n  author = {Juncheng Gu and Mosharaf Chowdhury and Kang G. Shin and Yibo Zhu and Myeongjae Jeon and Junjie Qian and Hongqiang Liu and Chuanxiong Guo},\n  title = {Tiresias: A {GPU} Cluster Manager for Distributed Deep Learning},\n  booktitle = {Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI)},\n  year = {2019},\n  pages = {485--500},\n  url = {https://www.usenix.org/conference/nsdi19/presentation/gu}\n}\n\n@techreport{ipp,\n  author = {Michael Sweet and Ira McDonald},\n  title = {Internet Printing Protocol/1.1: Model and Semantics},\n  institution = {RFC Editor},\n  number = {RFC 8011},\n  year = {2017},\n  doi = {10.17487/RFC8011}\n}\n\n@inproceedings{jeon2019,\n  author = {Myeongjae Jeon and Shivaram Venkataraman and Amar Phanishayee and Junjie Qian and Wencong Xiao and Fan Yang},\n  title = {Analysis of Large-Scale Multi-Tenant {GPU} Clusters for {DNN} Training Workloads},\n  booktitle = {Proceedings of the USENIX Annual Technical Conference (ATC)},\n  year = {2019},\n  pages = {947--960},\n  url = {https://www.usenix.org/conference/atc19/presentation/jeon}\n}\n\n@inproceedings{schroeder2006,\n  author = {Bianca Schroeder and Adam Wierman and Mor Harchol-Balter},\n  title = {Open Versus Closed: A Cautionary Tale},\n  booktitle = {Proceedings of the 3rd USENIX Symposium on Networked Systems Design and Implementation (NSDI)},\n  year = {2006},\n  url = {https://www.usenix.org/conference/nsdi-06/open-versus-closed-cautionary-tale}\n}\n\n@inproceedings{mahajan2020,\n  author = {Kshiteej Mahajan and Arjun Balasubramanian and Arjun Singhvi and Shivaram Venkataraman and Aditya Akella and Amar Phanishayee and Shuchi Chawla},\n  title = {Themis: Fair and Efficient {GPU} Cluster Scheduling},\n  booktitle = {Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI)},\n  year = {2020},\n  pages = {289--304},\n  url = {https://www.usenix.org/conference/nsdi20/presentation/mahajan}\n}\n\n@inproceedings{qiao2021,\n  author = {Aurick Qiao and Sang Keun Choe and Suhas Jayaram Subramanya and Willie Neiswanger and Qirong Ho and Hao Zhang and Gregory R. Ganger and Eric P. Xing},\n  title = {Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning},\n  booktitle = {Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI)},\n  year = {2021},\n  pages = {1--18},\n  url = {https://www.usenix.org/conference/osdi21/presentation/qiao}\n}\n\n@inproceedings{xiao2018,\n  author = {Wencong Xiao and Romil Bhardwaj and Ramachandran Ramjee and Muthian Sivathanu and Nipun Kwatra and Zhenhua Han and Pratyush Patel and Xuan Peng and Hanyu Zhao and Quanlu Zhang and Fan Yang and Lidong Zhou},\n  title = {Gandiva: Introspective Cluster Scheduling for Deep Learning},\n  booktitle = {Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI)},\n  year = {2018},\n  pages = {595--610},\n  url = {https://www.usenix.org/conference/osdi18/presentation/xiao}\n}\n","figures":[{"id":"sweep","kind":"chart","data":"{\"type\": \"line\", \"xlabel\": \"Jam probability per feed (%)\", \"ylabel\": \"95th-percentile wait (s)\", \"x\": [0, 2, 4, 6, 7, 8], \"series\": [{\"label\": \"Client\", \"y\": [121.23829987225581, 134.5959215148312, 159.82534019013622, 264.0330000596698, 560.5895038611174, 1758.537641011274], \"color\": \"C3\", \"marker\": \"o\", \"style\": \"solid\"}, {\"label\": \"Placement\", \"y\": [110.32246674039648, 113.58260146041765, 111.16659668927024, 108.02992470059961, 93.28882580377774, 69.81203633907927], \"color\": \"C0\", \"marker\": \"s\", \"style\": \"dashed\"}], \"legend\": \"upper left\", \"yscale\": \"log\", \"ylim\": [50, 2200], \"size\": [3.4, 2.4]}"},{"id":"outage","kind":"chart","data":"{\"panels\": [{\"type\": \"line\", \"xlabel\": \"Time from interruption (min)\", \"ylabel\": \"Queued requests\", \"x\": [-5.0, -4.5, -4.0, -3.5, -3.0, -2.5, -2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5, 10.0, 10.5, 11.0, 11.5, 12.0, 12.5, 13.0, 13.5, 14.0, 14.5, 15.0, 15.5, 16.0, 16.5, 17.0, 17.5, 18.0, 18.5, 19.0, 19.5, 20.0, 20.5, 21.0, 21.5, 22.0, 22.5, 23.0, 23.5, 24.0, 24.5, 25.0, 25.5, 26.0, 26.5, 27.0, 27.5, 28.0, 28.5, 29.0, 29.5, 30.0, 30.5, 31.0, 31.5, 32.0, 32.5, 33.0, 33.5, 34.0, 34.5, 35.0], \"ylim\": [-3, 210], \"series\": [{\"label\": \"Spool\", \"y\": [30.03125, 30.25, 30.84375, 30.5, 30.9375, 31.78125, 31.9375, 31.59375, 31.0625, 31.25, 31.65625, 36.4375, 41.21875, 45.90625, 50.6875, 55.75, 60.625, 65.71875, 71.0625, 75.46875, 80.21875, 84.4375, 89.53125, 94.21875, 98.84375, 103.46875, 108.34375, 113.3125, 117.8125, 122.46875, 127.15625, 131.65625, 135.96875, 140.03125, 145.34375, 150.09375, 154.6875, 159.46875, 164.75, 169.1875, 174.21875, 175.03125, 173.4375, 173.40625, 173.375, 172.75, 170.96875, 169.84375, 169.25, 168.1875, 167.46875, 166.65625, 165.03125, 164.0625, 164.15625, 163.34375, 162.53125, 160.8125, 159.5, 159.34375, 158.4375, 157.46875, 156.3125, 155.78125, 155.5625, 155.09375, 154.09375, 153.40625, 153.1875, 152.71875, 152.125, 152.53125, 152, 152.3125, 151.25, 151.15625, 152.375, 151.125, 152.40625, 151.625, 150.28125], \"color\": \"C3\", \"style\": \"solid\", \"marker\": \"none\"}, {\"label\": \"Placement\", \"y\": [2.15625, 2.3125, 2.25, 2.53125, 2.71875, 3.21875, 3.03125, 3.28125, 3.4375, 3.15625, 3.125, 1.90625, 1.25, 0.84375, 0.4375, 0.25, 0.03125, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.1875, 0.5, 0.28125, 0.25, 0.28125, 0.53125, 0.4375, 0.625, 0.65625, 0.84375, 0.65625, 0.59375, 0.78125, 0.90625, 0.84375, 0.9375, 1, 1.34375, 1.125, 1.34375, 1.46875, 1.40625, 1.8125, 1.875, 1.5, 1.28125, 1, 1.03125, 1.09375, 1.4375, 1.5, 1.375, 1.375, 1.40625, 1.59375, 2.25], \"color\": \"C0\", \"style\": \"solid\", \"marker\": \"none\"}, {\"label\": \"Spool, control\", \"y\": [30.03125, 30.25, 30.84375, 30.5, 30.9375, 31.78125, 31.9375, 31.59375, 31.0625, 31.25, 31.65625, 32.59375, 32.21875, 32.28125, 32.5625, 33.78125, 33.5, 33.625, 34.28125, 32.8125, 32.6875, 31.84375, 30.9375, 30.34375, 29.9375, 30.25, 30, 30.09375, 30.59375, 31.28125, 31.0625, 30.75, 29.90625, 29.78125, 30.40625, 30.71875, 29.28125, 29.78125, 30.5625, 30.71875, 31.15625, 32, 31.96875, 32.5, 32.0625, 31.5, 31.21875, 31.09375, 31.1875, 32.65625, 32.5, 33.3125, 34.25, 35.125, 35.15625, 35.03125, 35.1875, 34.34375, 34.375, 34.03125, 34.15625, 34.71875, 34.9375, 35.125, 35.34375, 35.09375, 35.3125, 35.15625, 35.65625, 35.5625, 35.59375, 36.125, 36.125, 36.9375, 36.28125, 36.75, 37.8125, 37.28125, 37.34375, 37.125, 36.65625], \"color\": \"C3\", \"style\": \"dashed\", \"marker\": \"none\"}], \"vlines\": [{\"x\": 0, \"color\": \"#646464\", \"style\": \"dotted\"}, {\"x\": 15, \"color\": \"#646464\", \"style\": \"dotted\"}], \"legend\": \"upper left\"}, {\"type\": \"line\", \"xlabel\": \"Time from interruption (min)\", \"ylabel\": \"Busy GPUs\", \"x\": [-5.0, -4.5, -4.0, -3.5, -3.0, -2.5, -2.0, -1.5, -1.0, -0.5, 0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5, 10.0, 10.5, 11.0, 11.5, 12.0, 12.5, 13.0, 13.5, 14.0, 14.5, 15.0, 15.5, 16.0, 16.5, 17.0, 17.5, 18.0, 18.5, 19.0, 19.5, 20.0, 20.5, 21.0, 21.5, 22.0, 22.5, 23.0, 23.5, 24.0, 24.5, 25.0, 25.5, 26.0, 26.5, 27.0, 27.5, 28.0, 28.5, 29.0, 29.5, 30.0, 30.5, 31.0, 31.5, 32.0, 32.5, 33.0, 33.5, 34.0, 34.5, 35.0], \"ylim\": [-1, 64], \"series\": [{\"label\": \"Interrupted\", \"y\": [53.53125, 52.21875, 51.25, 51, 51.4375, 50.875, 51, 52.53125, 53.15625, 52.15625, 53.09375, 46.40625, 39.65625, 32.84375, 28.5625, 25.71875, 23.03125, 19.46875, 16.875, 13.90625, 11.8125, 9.78125, 7.1875, 7.0625, 5.5, 4.75, 4.125, 3.375, 2.625, 1.875, 1.625, 1.125, 1, 1, 0.5, 0.5, 0.25, 0.25, 0, 0, 0, 9.90625, 20.5625, 28.65625, 34.9375, 37.78125, 42.78125, 45.5625, 46.8125, 49.75, 49.125, 48.5625, 48.90625, 49.46875, 46.90625, 47.0625, 46.96875, 48.34375, 50.09375, 48.90625, 48.78125, 49.4375, 51.6875, 51.84375, 51.125, 51.65625, 53.59375, 52.6875, 50.9375, 50.90625, 50.5625, 48.15625, 47.46875, 49.625, 52.0625, 52.75, 52.0625, 52.53125, 52.125, 52.59375, 54.21875], \"color\": \"C0\", \"style\": \"solid\", \"marker\": \"none\"}, {\"label\": \"Control\", \"y\": [53.53125, 52.21875, 51.25, 51, 51.4375, 50.875, 51, 52.53125, 53.15625, 52.15625, 53.09375, 52.59375, 53.8125, 52.09375, 51.46875, 50.09375, 52.09375, 53.375, 52.5625, 53.9375, 54.125, 54.78125, 56.34375, 56.25, 56.1875, 55.1875, 54.34375, 53.03125, 51.71875, 50.03125, 49.4375, 50.9375, 53.0625, 51.875, 53.09375, 53.5625, 56.25, 54.90625, 54.75, 54.65625, 54, 53.21875, 51.75, 53.59375, 54.28125, 54.28125, 52.96875, 50.96875, 52.0625, 50.8125, 52.4375, 52.65625, 51.84375, 52, 52.3125, 52.03125, 51.4375, 53.5, 53.0625, 53.28125, 53.25, 53.53125, 53.90625, 53, 54.28125, 54.46875, 54.1875, 54.09375, 53.21875, 53.125, 52.90625, 52.71875, 53.875, 52.03125, 51.03125, 51.15625, 50.1875, 50.28125, 52.0625, 53.40625, 52.40625], \"color\": \"C0\", \"style\": \"dashed\", \"marker\": \"none\"}], \"vlines\": [{\"x\": 0, \"color\": \"#646464\", \"style\": \"dotted\"}, {\"x\": 15, \"color\": \"#646464\", \"style\": \"dotted\"}], \"legend\": \"lower right\"}], \"size\": [6.8, 2.4]}"}],"options":{"citation_style":"numeric","ccs":["Computer systems organization → Cloud computing","Software and its engineering → Scheduling"]}}