[{"id":162,"title":"Learning computationally expensive functions","sha":"1670e0f0f5125e7292dec2f10f09ed59","user_id":349,"state":"published","body":"Analysing data from an experiment at the Large Hadron Collider at CERN requires large amounts of computing power.\r\n\r\nIn addition to large amounts of experimental data, each individual analysis requires large amounts of simulated data. This production of simulated data is the single largest consumer of CPU resources of the LHC computing grid [[Atlas Simulation](https://indico.cern.ch/event/279530/session/5/contribution/11), [LHCb Computing Resource usage in 2014](http://cds.cern.ch/record/1984010)]. Producing enough simulated data is already challenging and will become a limiting factor in the future with the size of experimental datasets poised to increase by O(10)--O(100).\r\n\r\nIn individual analyses sophisticated features are computed from basic quantities measured in the detectors. These sophisticated features improve the sensitivity of an analysis, but require a large amount of CPU time [[Mtop matrix element](http://arxiv.org/abs/hep-ex/0406031), [Measurement of the Single Top Quark Production Cross Section at CDF](http://arxiv.org/abs/0809.2581), [Matrix element for Higgs](http://arxiv.org/abs/1502.02485), [Missing mass calculator](http://arxiv.org/abs/1012.4686)]. The large CPU needs limit the number of cases in which these techniques are used, resulting in sub-optimal analyses.\r\n\r\nI propose replacing the calculations by a regression model [[Greedy Function Approximation: A Gradient Boosting Machine](http://www.jstor.org/stable/2699986), [Scikit-learn: Machine Learning in Python](http://jmlr.csail.mit.edu/papers/v12/pedregosa11a.html)] which is cheap to evaluate. The full simulation or computation of sophisticated features is performed on a subset of events, which are then used to train a regression model. This model is used to perform the computation for a large number of events. A proof of concept for the [Missing mass calculator](http://arxiv.org/abs/1012.4686) using [the dataset from the ATLAS Higgs Boson Machine Learning Challenge](http://doi.org/10.7483/OPENDATA.ATLAS.ZBP2.M5T8) is available: [Learning expensive functions](http://betatim.github.io/posts/learning-expensive-functions/).","created_at":"2015-05-18T11:39:18.249Z","updated_at":"2015-12-18T13:22:56.608Z","zenodo_id":29587,"doi":"http://dx.doi.org/10.5281/zenodo.17720","subject":null,"tags":["hep","cern","lhc","ml","machinelearning","regression","physics"],"vote_count":1,"view_count":1,"score":1.1,"deleted":false,"muted":false,"tweeted":true,"attachment_file_name":null,"attachment_content_type":null,"attachment_file_size":null,"attachment_updated_at":null},{"id":242,"title":"Modify and run other people's research code in your browser","sha":"9806c0ed3b27a42cf4be89374fbc6ae5","user_id":null,"state":"published","body":"Science makes progress by reusing results and building on them. For research software this is pretty hard (the people writing it often do not have the time to make slick installers like big libraries do). As a result there is not as much reuse as there could be. With [`everware`](//github.com/everware/everware) we are changing this.\r\n\r\nWith `everware` you can edit and run code from a git repository with one click, in your browser. This significantly reduces the barrier to entry for trying out other people's code on a whim. As a result you will try out and decide to reuse other people's code more often, instead of rolling your own. Furthermore, if reuse is possible, reproducibility comes for free.\r\n\r\nInterest in big discoveries like the Higgs boson is massive, imagine how many lay-people would love to be able to run (parts of) the analysis software that discovered the Higgs.\r\n\r\nAs the author of a research code all you have to do for your repository to be `everware`-ready is provide a `Dockerfile` that describes how to setup all the dependencies. Once this is done other's can launch your code from their browser and experiment with it.\r\n\r\n`Everware` builds on [github](//github.com), [docker](//docker.io) and [project jupyter](//jupyter.org). It started as a project at the [CERN webfest 2015](//webfest.web.cern.ch/). You can find [project everware](//github.com/everware/everware) on github.","created_at":"2015-09-29T20:59:58.263Z","updated_at":"2015-11-24T16:25:36.168Z","zenodo_id":62932,"doi":"http://dx.doi.org/10.5281/zenodo.31598","subject":null,"tags":["open research accelerator"],"vote_count":2,"view_count":1,"score":2.1,"deleted":false,"muted":false,"tweeted":true,"attachment_file_name":null,"attachment_content_type":null,"attachment_file_size":null,"attachment_updated_at":null},{"id":260,"title":"Brief Ideas for the Data Science at LHC Workshop 2015","sha":"48be7c9cc3de0c437011525e9d0b0180","user_id":null,"state":"published","body":"The Data Science @ LHC workshop was a resounding success. We do not plan to have traditional proceedings tied to individual talks, but we do want to capture the ideas that were generated during the workshop. With that in mind, we want to try something new: we encourage you to submit contributions to the Journal of Brief Ideas. Each contribution is capped at 200 words, so this provides a lightweight method of recording ideas that came up during the workshop. They do not have to be tied to a particular talk or speaker, and you can even submit several ideas with a different set of authors. The goal is to lay the groundwork for new collaborations and follow up studies, not to document completed research. As this is a new way of doing things, please let us know what you think or if need help with anything. Each publication will receive a DOI so that it can be cited by other publications and collected by INSPIRE as contributions to the conference.\r\nImportant : fill in your brief ideas by 22 January.\r\nVery important: please use this tag: DSLHC when composing the idea so that we can easily aggregate them.","created_at":"2015-12-22T22:54:51.146Z","updated_at":"2016-01-06T06:00:27.519Z","zenodo_id":92248,"doi":"http://dx.doi.org/10.5281/zenodo.44357","subject":null,"tags":["datascience","lhc","dslhc"],"vote_count":0,"view_count":1,"score":0.1,"deleted":false,"muted":false,"tweeted":false,"attachment_file_name":null,"attachment_content_type":null,"attachment_file_size":null,"attachment_updated_at":null},{"id":285,"title":"Create standalone simulation tools to facilitate collaboration between HEP and machine learning community","sha":"ff0489d51bdb17359cef823c1d6b7029","user_id":null,"state":"published","body":"Discussions at recent workshops have made it clear that one of the key barriers to collaboration between high energy physics and the machine learning community is access to training data. Recent successes in data sharing through the [HiggsML](http://doi.org/10.7483/OPENDATA.ATLAS.ZBP2.M5T8) and [Flavours of Physics](https://www.kaggle.com/c/flavours-of-physics/data) Kaggle challenges have borne much fruit, but required significant effort to coordinate.\r\n\r\nWhile static simulated datasets are useful for challenges, in the course of investigating new machine learning techniques it is advantageous to be able to generate training data on demand (e.g. Refs. [1](https://archive.ics.uci.edu/ml/datasets/HEPMASS), [2](https://archive.ics.uci.edu/ml/datasets/SUSY), [3](https://archive.ics.uci.edu/ml/datasets/HIGGS) ). \r\nTherefore we recommend efforts be made to produce the ingredients required to facilitate such collaboration:\r\n   * Specific challenges for HEP experiments should be fully specified such that minimal domain-specific knowledge is required to attack them.\r\n   * Stand-alone simulators should be made open source. They should be developed to be easy to use without domain-specific expertise, while still being representative of real experimental challenges. Such a simulation will permit non-HEP researchers to generate realistic HEP datasets for training and testing. These simulators could range from truth-level simulation of a hard scattering to fast simulation like [Delphes](http://dx.doi.org/10.1007/JHEP02(2014)057), to full [GEANT4](http://dx.doi.org/10.1016/S0168-9002(03)01368-8) simulation of sensor arrays.\r\n   * Performance metrics (objective functions) and operational constraints should be defined to evaluate proposed solutions.\r\n","created_at":"2016-02-26T21:34:14.721Z","updated_at":"2016-04-03T17:04:01.541Z","zenodo_id":99761,"doi":"http://dx.doi.org/10.5281/zenodo.46864","subject":null,"tags":["dslhc","machinelearning","datascience","open data","simulation"],"vote_count":3,"view_count":1,"score":3.1,"deleted":false,"muted":false,"tweeted":true,"attachment_file_name":null,"attachment_content_type":null,"attachment_file_size":null,"attachment_updated_at":null},{"id":294,"title":"Kickstarting research into end-to-end trigger systems","sha":"6e2cc286c54020f08d774487315585e8","user_id":null,"state":"published","body":"The data volumes (_**O**_(TB/s)) created at the Large Hadron Collider (LHC) are too large to record. Typical rejection factors are _**O**_(100-1000), and using as little CPU time as possible to reject an event is the goal. More powerful decision features take more CPU time to construct, therefore the discrimination power of decisions is limited by the available CPU time.\r\n\r\nCurrent approaches rely on hand-crafted features, and hand tuned decision cascades. We propose the possibility of using deep learning techniques to construct an end-to-end system (using the raw data electronics data as input) that learns the decision function of the trigger system (the global HLT decision). Deep Neural Networks can be efficiently evaluated on dedicated hardware and can be used to preempt the decision of the full system.\r\n\r\nFor LHCb we estimate that 750GB of data contain about 10000000 candidates of which 100000 will be accepted by the trigger system.\r\n\r\nTo kickstart research into this new area of research each of the four LHC experiments should release a few seconds of HLT output as well as the corresponding input. This will allow collaboration with experts from the field of machine-learning and cooperation between the experiments to solve this challenging problem.","created_at":"2016-04-01T11:31:52.210Z","updated_at":"2016-04-16T13:09:00.853Z","zenodo_id":105137,"doi":"http://dx.doi.org/10.5281/zenodo.48784","subject":null,"tags":["lhc","deep learning","machinelearning","data_exchange"],"vote_count":1,"view_count":1,"score":1.1,"deleted":false,"muted":false,"tweeted":true,"attachment_file_name":null,"attachment_content_type":null,"attachment_file_size":null,"attachment_updated_at":null}]