Dissecting Customized Protocols: Automatic Analysis for Customized Protocols based on IEEE 802.15.4

Kibum Choi, Yunmok Son, Juhwan Noh, Hocheol Shin, Jaeyeong Choi, Yongdae Kim

IEEE 802.15.4 is widely used as lower layers for not only well-known wireless communication standards such as ZigBee, 6LoW-PAN, and WirelessHART, but also customized protocols developed by manufacturers, particularly for various Internet of Things (IoT) devices. Customized protocols are not usually publicly disclosed nor standardized. Moreover, unlike textual protocols (e.g., HTTP, SMTP, POP3.), customized protocols for IoT devices provide no clues such as strings or keywords that are useful for analysis. Instead, they use bits or bytes to represent header and body information in order to save power and bandwidth. On the other hand, they often do not employ encryption, fragmentation, or authentication to save cost and effort in implementations. In other words, their security relies only on the confidentiality of the protocol itself.

In this paper, we introduce a novel methodology to analyze and reconstruct unknown wireless customized protocols over IEEE 802.15.4. Based on this methodology, we develop an automatic analysis and spoofing tool called WPAN automatic spoofer (WASp) that can be used to understand and reconstruct customized protocols to byte-level accuracy, and to generate packets that can be used for verification of analysis results or spoofing attacks. The methodology consists of four phases: packet collection, packet grouping, protocol analysis, and packet generation. Except for the packet collection step, all steps are fully automated.

Although the use of customized protocols is also unknown before the collecting phase, we choose two real-world target systems for evaluation: the smart plug system and platform screen door (PSD) to evaluate our methodology and WASp. In the evaluation, 7,299 and 217 packets are used as datasets for both target systems, respectively. As a result, on average, WASp is found to reduce entropy of legitimate message space by 93.77 % and 88.11 % for customized protocols used in smart plug and PSD systems, respectively. In addition, on average, 48.19 % of automatically generated packets are successfully spoofed for the first target systems.

Review:
This paper describes the design, implementation, and evaluation of WASp, a tool that automatically reverse-engineers proprietary wireless protocols that run over IEEE 802.15.4. Such protocols are commonly used in IoT systems, and two are studied in this paper: a smart plug system and a platform screen door system. Such protocols often depend on obscurity, because they 1) use proprietary binary encoding for the packets, 2) lack documentation, and 3) do not necessarily use encryption. Therefore it is a worthwhile goal to assess their security through reverse-engineering. The proposed WASp tool takes as input captured packets (pcap files) and some context information (e.g. number of nodes involved in the communication) and aims to first, reconstruct the packet format of the unknown protocol and then second, generate (“spoof”) packets that conform to the protocol.

The reviewers appreciated practical problem studied by the paper, noting that it will become more important as IoT deployments increase. The implementation and practical testing of the tool adds value and shows that the protocols studied can be reverse-engineered to some extent. The paper is in general well-written and easy to understand.

The main concern shared by all reviewers is that the evaluation of WASp does not use ground truth (i.e. reverse-engineer known protocols and evaluate WASp by comparing its output to the known packet format). The evaluation metrics used are the “spoofing success rate” (a spoofed packet is considered a success, if any response packet is detected for it) and the “entropy reduction” (this is not formally defined in the paper, but it relates to the reduction in the number of packets that the attacker – before and after running WASp – considers as potentially valid protocol packets). However, due to legal and safety reasons, the “spoofing success rate” cannot always be used, which was the case in the second system studied (platform screen door). Also, the entropy reduction method would benefit from a clearer definition and more justification as an evaluation metric. Overall, wider and more rigorous evaluation is needed, and this is a useful direction for future research.