detecting-attacks-web-applications-log-files

---

detecting-attacks-web-applications-log-files：

1. Abstract:

Web application log files allow a detailed analysis of a users actions.

Log files have its limits,though. Web server log files contain only a fraction of the fullHTTP request and response。

Knowing those limits, the majority of attacks can be recognized and acted upon to prevent further exploitation.

---

2. Introduction:

Why should you bother analyzing log files instead of using a network intrusion detection system?

reasons:

• The HTTP traffic may be SSL encrypted (HTTPS);
• There may be no NIDS (hard to deploy; another zone of
  attack);
• High traffic load makes it difficult to analyze network
  traffic (in real time);
• NIDS are designed to work on the TCP/IP level, and thus
  they may not be as effective on the HTTP layer;
• IDS evasion techniques (HTTP, encoding,
  fragmenting, …).

---

3. Attacks on Web Applications:

4个趋势：

• Bots are being used in more than half the attacks against
  web applications; 机器人
• Attackers are finding flawed web applications using Google and other search tools; 搜索
• Directed attacks are growing more sophisticated 复杂
• Attackers operating from bases around the world are
  getting better at covering their tracks 隐匿

---

3.1 Web server log files：

• standard web servers like Apache and IIS generate logging
  messages by default in the Common Log Format (CLF) specification.
• The CLF log file contains a separate line for each HTTP request.
• A line is composed of several tokens separated by spaces:

3.2 primer to http:

http (short for Hyper Text Transfer Protocol) **is the language
that web servers and browsers speak**.

• Universal Resource Locators or URLs:

• request和response例子：

3.2.1 HTTP Evasion Techniques:

• There are different types of evasions in different places of the
  HTTP protocol

• They can occur in the request URI portion of the
  HTTP protocol, other parts of the HTTP header, the HTTP body, etc

• 例子：加/

There could be infinite combinations of those evasions. The goal of these techniques is to evade the detection by an IDS but still get executed by the web application.

normalisation method:

• URL-decoding
  
  • %XX
• Null byte string termination
• self-referencing paths
  
  • /./
  • =号编码
• path back-references
  
  • /../
  • =号编码
• Mixed case
• Excessive use of whitespace
• Comment removal
  
  • DELETE/**/FROM
• **Conversion of (Windows­supported) backslash characters
  into forward slash characters**
• Conversion of IIS­specific Unicode encoding
  
  • %uXXYY
• Decode HTML entities
  
  • c, ", ª
• Escaped characters
  
  • \t, \001, \xAA, \uAABB

---

4.Detecting Attacks

• Layer 3/4 Firewall:

• Application layer firewall / web application firewall (WAF

• web server:

  • The web server is the end device of an HTTP request.
  • Standard web servers like Apache and IIS are logging by default in the
    Common Log Format (CLF) specification.

• Web server logs do not contain any data sent in the HTTP header, like POST parameters.

• The HTTP header can contain valuable data, as most forms and their parameters are submitted by POST requests. This comes as a big deficiency for web server log files

---

4.1 Rule­based Detection (static rules)

• Attacks can be detected by two different strategies: rulebased基于规则
  and anomaly­based. 基于异常

• The rule­based strategy defines static rules which have to be defined before the analysis can be made

• Static rules are defined once and stay the same during the detection phase.

• They have to be defined and specifically crafted for each application.

• Static rules make most sense for p**re­known values** like certain input characters, a fixed length of a parameter or an upper limit of a transfer amount.

Static rules can be further divided into two detection
models: the negative and the positive security model.

4.1.1 Negative Security Model:

The negative security model, or the blacklist approach, has a default policy of allow everything. This means that everything is allowed to pass, or everything is considered “normal”.

The policy (the blacklist, or the rule­base) defines what is not allowed.

• 劣势：

  • The biggest disadvantage is that the detection will **only be
    as good as the policy**. 能力取决于规则
  • It has to be adopted to new findings and updated to recognize new attack vectors需要根据新的攻击向量更新规则

• 好处：

  • One of the positive points is that it yields very little false­positives as the rules will usually look for specific, well known attack strings or behaviour.
  • 因为是用一些非常明显的特征来作为规则，所以误报率较小

sql注入中的用concat关键字匹配出现过误报：

---

4.1.2 Positive Security Model：

• 原则：

  • The default policy here is deny all, the policy will then define what is allowed.
  • everything else will be considered as malicious.

• 设置：

  • This whitelist **can be defined automatically in a learning phase **or be manually
    defined.

• 防火墙基于白名单：

  • Firewalls are usually configured this way. The default policy will be deny. For every server/service there has to be a new whitelist entry for this specific machine and port

• 从安全的角度推荐：

  • This model is the preferred way from a security standpoint. False negatives can be reduced to a minimum, while false positives help to improve the whitelist.误报率可以降到最低。

---

4.2 Anomaly­based Detection (dynamic rules)

• 原理：

  • Anomaly­rules consist of dynamic rules. As the name implies, those rules are** not static** nor are they manually defined. Instead, the rules are defined through a learning phase

• 定义正常的流量：

  • In this learning phase, good traffic is recorded as “normal“. It is of greatest importance, that this traffic is “clean“ and free of attacks, as this will be used as our baseline

• 识别异常流量：

  • The goal of a learning phase is to define how “normal“, accepted traffic looks like to eventually flag anomalous traffic which does not look like “normal“ and raise an alarm.
  • Deviations from this ruleset will be flagged as anomalous traffic.

---

4.3 Detecting the OWASP Top Ten:

全是正则~
粗略了浏览了一遍、暂时不细看了。。。

---

参考：

Detecting Attacks on Web Applications from Log Files