Week 1

Programming Languages

This is just a short update about what I've done so far. In my plan the first week was allocated to general planning and research. I looked into machine learning frameworks for the various languages I considered, focusing on Python, JavaScript, F# and Haskell. I've decided against F# and Haskell because it's unnecessary extra complexity: I want to minimise the number of new concepts I have to learn in this project to the bare essentials. LSTM networks are new to me, as is malware analysis in general, and I also need to learn a deep learning framework. F# is a new language, and I can't write Haskell as intuitively as I can the imperative languages I know. So I'm now between JavaScript and Python. I'm currently learning TensorFlow JS which provides a native code LSTM implementation that should be very fast (it can run on GPU). There is also TensorFlow for Python, but I would rather write JS all else being equal. The TensorFlow website says that the Python version is 1.5-2x faster than JavaScript in the browser [1], but it says nothing about using Node with the native or GPU binding. Since Node tends to be a lot faster than Python [2], I'd imagine Tensorflow JS in Node, without the browser overhead, is faster. (If I have time I'll implement a model in both and benchmark them myself.)

Books

I bought the book Practical Malware Analysis by Michael Sikorski. It's interesting and I've learned a few things about ways malware tries to evade detection, but I'm not sure how I'll apply this to the project yet. It might be best to encode these methods as features and have the model learn the features that correlate with a file containing malware.

Git Repository

I've created a Git repository on Github: https://github.com/ChrisSwinchatt/MalwareLearningMachine

References

Comments

Post a Comment

Popular posts from this blog

Plan

Time, Time, Time With the deadline falling on the 4th September 2018, there are roughly 12 weeks to complete the project from the time of writing. Treating this as a full-time job gives 40*12= 480 hours . The dissertation is 12,000 words; if it takes an hour to write and edit 100 words, then I will spend 120 hours (3 weeks) writing the dissertation, leaving 360 hours (9 weeks) to design & implement the software. I also won't count writing this blog against my 480 hours, as I can do it on the weekends. I'm going to spend the first week planning, reading, learning and prototyping. After that I will have 8 weeks of development time and three weeks of QA and dissertation-writing. Project Goal and Breakdown Goal of the project Predict whether a given executable file contains malware. Steps Predict whether a binary contains a valid program  Disassemble a binary and produce valid disassembly (this is the minimal viable product) Predict whether the disassembled prog...
Read more

Weeks 2-4

Technical Issues! So, first, I've completely deviated from my planned posting schedule due to a series of somewhat embarrassing technical problems I've experienced. Everything from difficulties building frameworks due to dependency hell, to realising that they either won't do what I want to do with them or lack documentation and provide examples which don't compile, to Windows kernel bugs and hosing my Linux distro by switching from the Stable branch to Testing. Programming Languages Continued Near the end of the second week, I still hadn't fully settled on the language. After speaking with Martin about my difficulties with F# and JavaScript, and my general dislike of Python, I decided, with Martin's encouragement, to try Haskell. I learned Haskell during my undergraduate degree and found it a very expressive language with a reasonable amount of library support at a very high (i.e. abstract) level. Enough to write my undergraduate dissertation project, an ...
Read more