Everything, Everything - March 2016

2018: J F M A M J J A S O N D
2017: J F M A M J J A S O N D
2016: J F M A M J J A S O N D
2015: J F M A M J J A S O N D
2014: J F M A M J J A S O N D
2013: J F M A M J J A S O N D
2012: J F M A M J J A S O N D
2011: J F M A M J J A S O N D
2010: J F M A M J J A S O N D
2009: J F M A M J J A S O N D
2008: J F M A M J J A S O N D
2007: J F M A M J J A S O N D
2006: J F M A M J J A S O N D
2005: J F M A M J J A S O N D
2004: J F M A M J J A S O N D
SMB Protocol, Versions, And Badlock
Thursday 31st March, 2016 16:53
With a patch and details being disclosed in the next couple of weeks for the Badlock bug, and a recent query from a friend about unauthenticated SMB requests, I thought it'd be useful to collate my knowledge of SMB.

SMB

The original SMB protocol was introduced a long, long time ago by Microsoft. Their original implementation for Windows NT 4.0 is actually called CIFS, but to keep things simple I will call it all SMB (or SMB 1). This was a proprietary protocol, but the Samba team created an implementation of SMB for UNIX, and it's now very well supported on Linux and BSD. SMB was originally designed to run on top of the NetBIOS/NetBEUI API (NBF, NetBIOS over IPX/SPX, or NBT). Since Windows 2000, SMB uses "direct host SMB" running over TCP port 445 rather than TCP port 139.

There were subtle differences between SMB and Samba, with Samba reporting version numbers that were lower than the OS version returned by Windows hosts in an attempt to avoid winning elections; but for file sharing it essentially behaved the same. Versions before Samba 3.5 only supported SMB 1, and could only be domain members (not domain controllers).

SMB 2

When people talk about SMB 2 they are probably talking about SMB 2.x and 3.x. Also, versions of SMB 2 before 2.0.2 are already obsolete!

Microsoft introduced SMB 2 with Windows Vista. Although this was another proprietary protocol, the specification was published to allow other systems to interoperate, making it easier for Samba to develop support (although it still took them years). The first release with experimental support for SMB 2 was Samba 3.5, with full support as of Samba 3.6. With lots of users still clinging onto Windows XP, and/or hating Vista's UAC and huge memory requirements, I suspect very few people were that concerned.

SMB 2 offered several major improvements over SMB 1 such as:
  • Reduced complexity - from over 100 commands and subcommands to just 19
  • Request compounding - allows for sending multiple SMB 2 requests as a single network request
  • Larger reads and writes - making better use of faster networks, even with high latency
  • Caching of folder and file properties - clients keep local copies of folders and files
  • Durable handles - allow for connection to transparently reconnect to the server if there is a temporary disconnection
  • Improved message signing - HMAC SHA-256 replaces MD5 as hashing algorithm
  • Improved scalability for file sharing - number of users, shares, and open files per server greatly increased
  • Support for symbolic links
  • Client oplock leasing model - limits the data transferred between the client and server, improving performance on high-latency networks and increasing SMB server scalability
  • Large MTU support - for better use of 10GB Ethernet connections
  • Improved energy efficiency - clients that have open files to a server can sleep
  • Better support for NAT - VC count was removed)
SMB 2 no longer supports NetBIOS over IPX, NetBIOS over UDP or NetBEUI. SMB 2 uses the existing SMB 1 connection setup mechanism, and advertises that it is capable of a new version of the protocol, which is covered in more detail further down.

SMB 2 allows clients to build a pipeline of requests instead of waiting for a response before sending the next request, which improves performance using a high latency network. It uses a credit based flow control, allowing the server to control a client's behaviour. The server starts with a small number of credits and scales up as needed to better utilize available bandwidth.

The reduced complexity in the command set for the SMB 2 protocol was accomplished by allowing an arbitrary set of commands to be compounded in a single request, allowing it to mimic SMB 1 without the complexity of a larger command set (e.g. the RENAME command can be replaced with three compounded SMB 2 commands). These compounded commands can even be unrelated. This improves performance due to the reduced number of network round trips.

There are apparently four types of opportunistic locks:

Batch Locks
To mitigate performance issues associated with repeatedly opening and closing a file in a short period of time, a client may ask for an Oplock of type "batch". The client delays sending the close request, and if a subsequent open request is given, the two requests cancel each other.

Exclusive Locks (SMB 2 only)
If a file is only opened in "shared mode" from an SMB server, the client receives an exclusive Oplock from the server. The client can safely assume that it is the only process with access to the file, allowing the client to cache all changes before committing them to the server, which improves performance. If another client tries to open the same file, the server sends a "break" request to the client which invalidates the exclusive lock. The client then flushes all changes to the file.

Level 2 Oplocks
After a client has relinquished its exclusive Oplock, to allow another client to have write/read access, the original client may then receive a "Level 2 Oplock" from the server. This allows the caching of read requests, but excludes write caching.

Filter Locks
A filter opportunistic lock locks a file so that it cannot be opened for either write or delete access. All clients must be able to share the file. Filter locks allow applications to perform nonintrusive filtering operations on file data (for example, a compiler opening source code or a cataloging program). A filter opportunistic lock differs from a level 2 opportunistic lock in that it allows open operations for reading to occur without sharing-mode violations in the time span between your application's opening the file and receiving the lock.

An Oplock is broken in different ways, depending on whether you're using SMB 1 or SMB 2:
  • For SMB, the server sends an Oplock break to the client using SMB_LOCK_ANDX. The LockType field is the level that the Oplock is broken into.
  • For SMB 2, there is a specific command (SMB2_OPLOCK_BREAK,0x12) used to send the Oplock break notification to the client.
There are a few registry keys that control opportunistic locking, but these are only valid for SMB 1. For example:

To disable Oplocks on a Windows client:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MRXSmb\Parameters
OplocksDisabled = 1

To disable Oplocks on a Windows server:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
EnableOplocks = 0

You cannot turn off opportunistic locking for SMB 2.

Unfortunately, SMB 2 also had a pretty nasty security flaw that was addressed in MS09-050. This was the SMBv2 Negotiation Vulnerability (CVE-2009-3103). Public exploit code existed, but the original proof-of-concept only resulted in a denial of service. In later weeks and months, this would eventually become a fairly reliable exploit.

I'm fairly sure I read on an old Microsoft blog that SMB 2.0.2 will fall back to SMB 1 if an older version of SMB 2 (i.e. 2.0.1) is negotiated. This is backed up by Microsoft's latest documents, which state that SMB 2 must have certain SMB 2 header values. For example, the DialectRevision must be set to 0x0202 and the dialect string must be set to "SMB 2.002" in order to negotiate support for SMB 2.0.2.

SMB 2.1 and SMB 3.x

The SMB server scans the dialect provided by the client for the string "SMB 2.???". The DialectRevision must also be set to at least 0x02FF (I believe that later versions of SMB have the DialectRevision of 0x0300, 0x0302, and 0x0311).

Just like how Windows 7 is actually version 6.1 behind the scenes, SMB 3.0 was originally developed as SMB 2.2 and was rebranded as SMB 3.0 when Windows 8 and Windows Server 2012 were released. Windows 10 and Server 2016 currently use SMB 3.1.1; I wouldn't expect to see SMB 4.0 anytime soon.

If the SMB server does not implement the SMB 2.1 or 3.x dialect family, it follows the old 2.0.2 negotiation process mentioned above. Microsoft produced a nice diagram in their SMB 2 documentation to show this negotiation:

SMB Negotiation Process

SMB 2.1 is essentially the same as SMB 2.0, but there are some minor performance enhancements and a new opportunistic locking mechanism. This "client Oplock leasing model" can improve performance by controlling caching of network files by the client. The primary types of leases available are:
  • Read-caching lease: allows caching reads and can be shared by multiple clients.
  • Write-caching lease: allows caching writes and is exclusive to only one client.
  • Handle-caching lease: allows caching handles and can be shared by multiple clients.
SMB 3.0 can be negotiated with Windows 8 and Server 2008, and provides the following benefits that are particularly useful for Hyper-V:
  • Transparent Failover - clients reconnect without interruption to cluster nodes during maintenance or failover
  • Scale Out – concurrent access to shared data on all file cluster nodes
  • Multichannel - aggregation of network bandwidth and fault tolerance if multiple paths are available between client and server
  • SMB Direct – adds RDMA networking support for very high performance, with low latency and low CPU utilization
  • Encryption – Provides end-to-end AES-128-CCM encryption
  • Directory Leasing - Improves application response times through caching
  • Performance Optimizations - for small random read/write I/O
SMB 3.0.2, introduced with Windows 8.1 and Windows Server 2012 R2, is very similar to SMB 3.0, but optionally allows SMB 1 to be disabled to increase security (because SMB 1 is essentially still a mess of legacy code).

SMB 3.1.1 was introduced in Windows 10, and will be used in Windows Server 2016. It makes secure negotiation mandatory when connecting to clients using SMB 2.x and higher (it could be disabled in older versions because "some third-party implementations of SMB did not correctly perform this negotiation", which probably means Samba's implementation?), and adds optional support for AES-128-GCM encryption (and signing with AES-CMAC). It turns out that AES-128-GCM performs better in most modern processors, and can double the speed of large file transfers.

Unauthenticated Requests

Something that was flagged by Daniel Miller on Twitter, and verified by myself, around Christmas 2015 was that Nmap's smb-os-discovery script wasn't getting the same amount of detail back from versions of Windows Server, but the full information was returned from desktop OS such as Windows 7. When valid credentials are supplied, all versions of Windows returned the full information. However, when using NULL authentication, which the Nmap script (and attackers) will generally use, for some reason the servers don't return as much information. This suggests there's an inconsistency in the way that Windows Server generates the response, which may be down to tighter group policy settings, but is possibly just an undocumented quirk.

However, it's been suggested that you can get this information from SMB 2 if you send a session setup request using NTLMSSP (NTLMSSP_Negotiate). When the server responds with an NTLMSSP_Challenge the packet will includes information such as the OS, DNS, Domain, time. Hopefully Nmap will have better results on newer systems once the script has been updated.

SMB Versions

This table shows the version of SMB that will be negotiated between two hosts, assuming that everyone's running a fully patched version of Windows with default configurations.
OS10 / 20168.1 / 2012 R28 / 20127 / 2008 R2Vista / 2008Previous
10 / 2016SMB 3.1.1SMB 3.0.2SMB 3.0SMB 2.1SMB 2.0.2SMB 1.x
8.1 / 2012 R2SMB 3.0.2SMB 3.0.2SMB 3.0SMB 2.1SMB 2.0.2SMB 1.x
8 / 2012SMB 3.0SMB 3.0SMB 3.0SMB 2.1SMB 2.0.2SMB 1.x
7 / 2008 R2SMB 2.1SMB 2.1SMB 2.1SMB 2.1SMB 2.0.2SMB 1.x
Vista / 2008SMB 2.0.2SMB 2.0.2SMB 2.0.2SMB 2.0.2SMB 2.0.2SMB 1.x
PreviousSMB 1.xSMB 1.xSMB 1.xSMB 1.xSMB 1.xSMB 1.x

Disabling and Enabling SMB Protocols

More information is available here from Microsoft, but you can use sc.exe to reconfigure the relevant service (from an elevated command prompt, and a reboot is required). For example:

To disable SMB 1 on the SMB client, run the following commands:
sc.exe config lanmanworkstation depend= bowser/mrxsmb20/nsi
sc.exe config mrxsmb10 start= disabled

To enable SMB 1 on the SMB client, run the following commands:
sc.exe config lanmanworkstation depend= bowser/mrxsmb10/mrxsmb20/nsi
sc.exe config mrxsmb10 start= auto

To disable SMB 2 and SMB 3 on the SMB client, run the following commands:
sc.exe config lanmanworkstation depend= bowser/mrxsmb10/nsi
sc.exe config mrxsmb20 start= disabled

To enable SMB 2 and SMB 3 on the SMB client, run the following commands:
sc.exe config lanmanworkstation depend= bowser/mrxsmb10/mrxsmb20/nsi
sc.exe config mrxsmb20 start= auto

PowerShell

In Windows 8 or Windows Server 2012, there is a PowerShell cmdlet that can help determine what version of SMB the client has negotiated. This is called Get-SmbConnection, or "gsmbc" for short. After issuing a command to retrieve a file (or directory listing) from an SMB server, you have about 10 seconds to execute the cmdlet. To test it against yourself, run the following PowerShell commands from an elevated PowerShell:

PS C:\> dir \\localhost\c$

PS C:\> gsmbc -ServerName localhost

The Dialect column will reveal the version that was negotiated, which is 3.1.1 on my Windows 10 host.

You can also enable auditing of SMB 1 traffic using PowerShell, to create events if clients are still using SMB 1.

To enable auditing of SMB 1 traffic, use the cmdlet:

Set-SmbServerConfiguration –AuditSmb1Access $true

To view the SMB 1 events, use the cmdlet:

Get-WinEvent -LogName Microsoft-Windows-SMBServer/Audit

You can also uninstall SMB 1 on modern versions of Windows (that support SMB 3.0.2 or higher?) using the cmdlet:

Remove-WindowsFeature FS-SMB1

Badlock

I've updated and published this post now that details of Badlock have been released, with more details about what appears to be a man-in-the-middle issue that can be mitigated by following best practice recommendations from the last decade of enforcing SMB signing and requiring NTLMv2 (or ideally disabling it completely and only using Kerberos).

It's a real anti-climax. I thought, given the hype and its name, it might be related to the LockFile and/or LockFileEx functions that lock a specified range of bytes in a file. According to Microsoft's own documentation, the range "may extend beyond the current end of the file". I thought that Badlock's use of the Heartbleed design on their site might be a clue that it was a similar issue where an attacker requests a bad range, resulting in a buffer over-read. I wondered if this range was copied into (kernel) memory on the server and the range exceeds the actual file size (or can be used to access arbitrary memory) then could it return kernel data? Or could it allow arbitrary kernel memory to be overwritten when flushing data, e.g. when the lock is broken? It turns out it wasn't any of that.

This isn't the first time that someone may be able to gain privileged access on hosts due to a vulnerability in SMB, but this isn't the most exciting one either. Older versions of Samba allowed anonymous users to gain root access through an error in Samba's RPC; and as mentioned above Windows Vista had a nasty vulnerability that allowed unauthenticated users to gain SYSTEM privileges on a host. This is the first time that someone's given the vulnerability a decent name and logo, hence the publicity.

With most organisations using Windows Update (or WSUS, or other patch management software) to regularly deploy security fixes, I suspect most Windows users will be protected within a matter of hours or days.

Sources

@RobNicholls81
Thursday 31st March, 2016 03:35
Resizing SVG With Internet Explorer https://t.co/f5OXNlg4cz
@RobNicholls81
Wednesday 30th March, 2016 20:40
More details on the horror that is Ubuntu on Windows: https://t.co/Nbc12UTeKM
@RobNicholls81
Wednesday 30th March, 2016 20:36
Initially sounds interesting, but might be an entire subsystem (like POSIX) rather than something less invasive? :S https://t.co/eBMtcvSeFJ
Resizing SVG With Internet Explorer
Wednesday 30th March, 2016 17:59
My code used to work in every browser. It used to work fine in Internet Explorer. It's stopped working in IE11 and Edge.

After a bit of fiddling, I found a very simple workaround. I saved the SVG file as PNG and modified the HTML to load the PNG version. Now it all works fine, although the PNG resizing using JavaScript isn't as smooth or pretty as the SVG version (which was why I went with SVG to begin with). I think it's safe to blame IE for not displaying SVG files properly, and I'm wondering if there's been a change in behaviour.
@RobNicholls81
Monday 28th March, 2016 18:45
Vote for Wotsits, we can't have Monster Munch Win the World Cup of Crisps 2016! https://t.co/igymk9cdBn
@RobNicholls81
Saturday 26th March, 2016 11:42
Pentesters, why not use this bank holiday weekend to update your CV and send it to @CTXIS. https://t.co/w78L63Lxmx https://t.co/vV1iWOB1pW
@RobNicholls81
Saturday 19th March, 2016 14:13
After reviewing hundreds of CVs over the last few years, yesterday I saw a CV submitted as a JPEG. Not even PNG or GIF. Stick to PDF!
@RobNicholls81
Saturday 19th March, 2016 13:29
Iwobi showing why he should be a first team regular for Arsenal. An impressive first half by the team, for a change.
@RobNicholls81
Saturday 19th March, 2016 12:53
Such good movement, and a nice finish. Great start for Arsenal.
@RobNicholls81
Wednesday 16th March, 2016 22:10
Summer Signings https://t.co/sLvc42ceEw
Summer Signings
Tuesday 15th March, 2016 22:43
Assuming Arsene doesn't resign, he'll need to address the concerns raised by the fans over the years. We need a striker. We need a defensive midfielder. With Arteta and Flamini reaching the end of their contracts, we desperately need backup or competition for Coquelin. My best guess for DM would be Grzegorz Krychowiak, as he's at a club that would probably sell for a decent price, he's the right age, he's friendly with Szczesny, sponsored by Puma, and it doesn't hurt that he's quite tall.

Identifying a striker is much harder. Alexandre Lacazette seems most likely out of all the top strikers. He's open to playing in the Premier League (unlike some other names), is relatively young, we have French players and a manager to support him, and Lyon seems willing to sell him for a reasonable price (based on them accepting £22m from Newcastle).

Personally, I'd look to sell some players that haven't progressed in order to help fund these transfers (although there's plenty of money in the bank). Oxlade Chamberlain and Walcott should be below Campbell in the pecking order (but no one will match Walcott's salary so we're probably stuck with him). Although Wilshere is regularly injured, a deeper role might suit him better (he seems to play deeper for England, and would be similar to how Cazorla has dropped deeper and brings the ball forward now). Ramsey has talent, but I don't think our current formation and tactics suit him, which might be why he's most effective on the right (with freedom to roam). With Iwobi continuing to impress, I do wonder how Ramsey and Iwobi best fit into the team. It might work better with a 4-1-4-1 formation with Iwobi (Or Ramsey) and Ozil in the middle, Campbell and Sanchez on the wings (swapping sides), and a top striker up front (Welbeck and Giroud have talent, but they're not someone like Suarez or Messi or Lewandowski or other strikers that regularly create goals out of half chances).

Hopefully we won't end up signing just one player over the summer, otherwise it'll be another year without any progress.
@RobNicholls81
Thursday 10th March, 2016 23:10
Android N mostly working. Difficult to retweet when buttons don't appear in Twitter. Instagram briefly showed an error but appears to cope.
@RobNicholls81
Tuesday 8th March, 2016 19:39
Walcott with two terrible attempts at a corner kick. Hopefully the rest of the team won't follow the example of our captain.
@RobNicholls81
Sunday 6th March, 2016 14:20
Storage Spaces Dual Parity https://t.co/Z6SgaSCl3Z
@RobNicholls81
Sunday 6th March, 2016 10:54
Storage Spaces Dual Parity https://t.co/Z6SgaSkJFp
Storage Spaces Dual Parity
Sunday 6th March, 2016 10:18
I stumbled across this message in a TechNet forum and this may explain why write performance is relatively slow, and why you need so many disks to enable dual parity (and why disk space seems to disappear):

Storage spaces uses erasure coding for its dual parity scheme, which optimizes recovery for the common case (single disk failure). This comes at the cost of higher overhead, which is 3 columns of "parity" information instead of the traditional 2. So for a 7 disk, 7 column dual parity space the amount of usable capacity is (7-3)*disk size, so you get 8TB with 7x2TB disks.

I get the impression that Storage Spaces is only of (limited) benefit to some home enthusiasts, some medium sized businesses, and perhaps large business that solely use Microsoft products. It sounds to me that the flexibility and easier support doesn't necessarily outweigh the benefits of software or hardware RAID setups. Which is a shame as I'd quite like to use it instead of doing rather more rigid software RAID5 in Windows.
@RobNicholls81
Saturday 5th March, 2016 14:33
Noooo. BT Sport has lost its own stream!
@RobNicholls81
Saturday 5th March, 2016 10:42
Life After Wenger https://t.co/NYlYgrF1yh
Life After Wenger
Saturday 5th March, 2016 10:40
After several poor results, it looks like many Arsenal fans are finally losing faith in Arsene Wenger. His lack of activity in the transfer windows, his unusual substitutions, the occasionally absurd reasons for poor performances and results. I can understand why many are calling for him to resign. But who do you bring in to replace him?

In the past, people have suggested Jurgen Klopp or Pep Guardiola. Both are big personalities with good records, and fans would probably be happy with such an appointment. But these managers, and several other candidates, have recently joined or are about to join new clubs.

Other suggestions include managers such as Joachim Low. Again, he's got a good reputation, he's worked with the Germans in our team, and the fans would like him and the type of football he'd bring. But he signed a contract last year to continue managing the German team until 2018. I can't see him leaving early unless they have a disastrous Euro 2016.

That doesn't leave many options in the short term. We could risk bringing in someone younger, less experienced, but with a good track record. With other managers still finding their feet, and in some cases needing to replace many players in their first team, it may even be the best time to bring in some fresh blood at the end of this season. Our own team is okay, it just needs a few great additions, and it appears we have the money in the bank to support that. That would give an ambitious manager a great team to try and beat established names as they struggle with their new clubs and/or new teams in an unfamiliar and ultra-competitive Premier League.

After doubts over the years, I'm still on the fence whether Arsene should stay or go; it really depends on who the alternative is. But there are plenty of managers out there that could be given the chance. Hopefully the board have a good idea, as a weak finish to this season (particularly if we don't retain the FA Cup) could see the fans calling for Arsene to move on.

Anyway, it's the day of the North London Derby, a day to get behind the entire team and the manager, and hope that we can confidently beat Spurs in the first step to getting our mojo back. COYG.
@RobNicholls81
Wednesday 2nd March, 2016 20:19
I probably shouldn't have joked earlier that Swansea would equalise, we'd push hard for the win, and they'd counterattack and win 2-1.
@RobNicholls81
Wednesday 2nd March, 2016 16:41
My old method of automatically tweeting my blog posts has stopped (unless I pay money) so I'll manually post them for now.
© Robert Nicholls 2002-2018
The views and opinions expressed on this site do not represent the views of my employer.
HTML5 / CSS3