boot0md: dual boot Windows 7 and Mac OSX from Win7 drive

[dmazar]

Since boot0 and variants need to fit into 440 bytes of MBR it's good to know where to find more space if it would be needed for future requirements. Since I'm messing with boot0 currently, I'll write some ideas. Do not know where to put it, so I'll put it here. May be of some use to somebody in the future.



Shrinking of "user interface"
--------------------------
I'll try not to go too deep into the code here to make this useful to decision makers who do not need to know coding details.

boot0 writes some messages if compiled in VERBOSE mode which is default:
boot0: GPT - after positive check fo GPT and right before scanning GPT array for HFS+ partition
boot0: test - right before checking HFS+ partition for HFS+ signature
boot0: done - after bootable partition is found and before passing control to it's boot loader
boot0: error - if no bootable partition is found

Some space may be made free by removing or shrinking those messages. I'll list some examples here to help get some feeling about what can be done.

Example 1: setting VERBOSE=0 leaves only "boot0: error" message
= releases 32 bytes

Example 2: removal of "boot0: GPT" and "boot0: test" messages
and leaving done and error messages
= releases 21 bytes

;LogString(gpt_str) ; 6 bytes
;LogString(test_str) ; 6 bytes
...
;gpt_str         db  'GPT', 0 ; 4 bytes
;test_str      db  'test', 0 ; 5 bytes

= releases 6 + 6 + 4 + 5 = 21 bytes

Example 3: extreme: no messages at all by removing all "UI" code and data
= releases 84 bytes in current boot0 (19% of the total 440 bytes space).

Example 4: remove all messages and just write character '-' in case of error
by removing all UI code except print_char and using it to print '-'
= releases 69 bytes


I think that playing with "UI" is the most easier and most effective way of getting more space. This should be considered first if more space for code would be needed.

[dmazar]

GPT disk check
--------------
I'll need to try to explain this with some pseudo code. Do not know how to do it without that.

Currently, checking if disk is GPT works like this:

Code: [Select]

// following is part of MBR partition table scanning
// iterating over MBR records:
BL = 0; // flag if protective MBR entry (type 0xEE) is found
for i = 0 to 3 {
    ...
    // if protective entry is found then set it inactive
    // and set flag BL = 1
    if (partition[i].type == 0xEE) {
        partition[i].flag = inactive;
        BL = 1;
    }
    ...
}
// if protective MBR entry was found then call checkGPT()
if (BL == 1) {
    checkGPT();
}
...
// checkGPT()
// checks if sector 1 starts with "EFI PART"
// and then scans GPT partitions
checkGPT() {
    if (sector 1 does not start with "EFI PART") {
        return;
    }
    ...
}

It seems to me that checking for "EFI PART" in sector 1 is enough and the code could be made smaller by removing BL flag. This means checkGPT would be called even on pure MBR disk, but there is no harm in this - "EFI PART" will not be found.

Code: [Select]

// MBR partition table scanning
for i=0 to 3 {
    ...
    if (partition[i].type == 0xEE) {
        partition[i].flag = inactive;
    }
    ...
}
// always call checkGPT(), it will do it's own checking
// no harm in doing that check even on pure MBR disk
checkGPT();
...
// and checkGPT() as before
checkGPT() {
    if (sector 1 does not start with "EFI PART") {
        return;
    }
    ...
}

this releases 6 bytes.

And, if we assume that in normal setups protective MBR entry will always be inactive (it's nonsense to put it active, but this should be investigated more), then code can be further reduced:

Code: [Select]

// MBR partition table scanning
for i=0 to 3 {
    ...
    // no MBR protective enrty checking at all
    ...
}
// always call checkGPT(), it will do it's own checking
checkGPT();
...
// and checkGPT() as before
checkGPT() {
    if (sector 1 does not start with "EFI PART") {
        return;
    }
    ...
}

this releases 15 bytes in total.

[KillerJK]

This is a minor thing, as I suppose boot0 and boot0hfs will eventually be replaced by boot0md, but anyway...

In boot0hfs.s, line 357

Code: [Select]

.switchPass2:
    ;
    ; Switching to Pass 2
    ; try to find a boot1h aware HFS+ MBR partition

That's good for boot0.s. Pass2 in boot0hfs.s is however checking for an active partition

Also, if you really really need a couple of extra bytes after all those optimizations, some jmp can be changed by a jmp short. This is automatically done by the assembler if you use -Ox. There are some problems with this because xcode has an obsolete assembler, different versions use different default arguments and -O2 in the version used by xcode has a bug that was preventing it from successfully assembling boot0. If you ever need it I documented those things in the .swithPass2 patch http://forum.voodooprojects.org/index.php/topic,1813.0.html.

So, because the optimization flag wasn't in the makefile, I added a few shorts and some other changes. My lazy way of finding jmps I could replace was comparing the optimized and unoptimized hex dumps, and also moving code around.

[dmazar]

Hi KillerJK. Thanks. I've seen your posts - made me realize that one jump in boot0md was not optimized. That was fixed in the "work" version.

Regarding optimizations and my previous post:

And, if we assume that in normal setups protective MBR entry will always be inactive (it's nonsense to put it active, but this should be investigated more), then code can be further reduced:

It seems that part can not be removed:
http://forum.voodooprojects.org/index.php?topic=572.0

[dmazar]

I wanted to finish what was started and done some tests with boot0work code. It does it's job as planned, although I do not have enough experience to judge if this would cover all installations. I'll leave it to you guys.

New src and bin attached. Different debugging added. If compiled with DEBUG=0 gives the same code as previous version. DEBUG=1 gives the code with debug output (info in the src).


Test cases: 
-----------

A) boot0md functionality

Disk0: MBR (boot0work), P1=Win7 (NTFS, bootable, active)
Disk1: GPT (boot0work), P1=EFI, P2=MacOSX Lion (HFS+, boot1h, boot), P3=MacOSX Lion Inst (HFS+, boot1h, boot), P4=MacOSX SL (HFS+, boot1h, boot)

TC 1: boot from Disk0, loads Cham. from Disk1/P2, can boot all, Win7 hybrid sleep works - OK
debug: P Dptppp P Dptppp DptpppGll+ => Pass2, GPT found, loading from second GPT partition

TC 2: Disk1 removed, boot from Disk0, loads Win7 from Disk0/P1, Win7 hybrid sleep works - OK
(to test what would happen with Win7 if Cham is not available any more)


B) boot0 functionality: search active HFS+ on boot disk only
note 1: orig. boot0 searches for any active partition. boot0work can not do that since this would break boot0md functionality
note2: tests were done by loading Chameleon installs from USB disk and loading OSes from other disks. USB partitions contained only Cham. and not full OSes. I was testing if stage 0 code can find valid partition to load stage 1 and stage 2. OS loading is stage 2 job, anyway.

Disk0 and Disk1 as above, plus
USB Disk: MBR (boot0work), P1=exFAT, P2=HFS+ (boot1h, boot), P3=HFS+ (boot1h, boot)

TC 3: USB/P3 set active, boot from USB, loads Cham. from USB/P3, can boot all - OK
debug: P Dptptptl+ => Pass1, loading active P3

TC 4: USB/P2 set active, boot from USB, loads Cham. from USB/P2, can boot all - OK
debug: P Dptptl+ => Pass1, loading active P2


C) boot0hfs functionality: find first bootable HFS+
disk setup as in B)

TC 5: USB/P1 set active, boot from USB, loads Cham. from USB/P2, can boot all - OK
debug: P Dptptptp P Dptptl+ => Pass2, loading first bootable HFS+, USB/P2

TC 6: boot from Disk1, loads Cham. from Disk1/P2, can boot all - OK
debug: P DptpppGll+ => Pass1, GPT found, loading from first HFS+ bootable partition


D) boot from EFI

TC 7: USB Disk: GPT (boot0work), P1=EFI (boot1f32, boot), P2=HFS+ (not bootable), P3=HFS+ (not bootable)
USB formatted to GPT, Cham. added to EFI, boot from USB, loads Cham. from USB/P1 EFI, can boot all - OK
debug: P DptpppGl+ => Pass1, GPT disk, loading from EFI

TC 8: USB Disk: GPT (boot0work), P1=EFI (HFS+, boot1h, boot), P2=HFS+ (not bootable), P3=HFS+ (not bootable)
USB formatted to GPT, EFI formatted to HFS+, Cham. added to EFI, boot from USB, loads Cham. from USB/P1 EFI, can boot all - OK
debug: P DptpppGl+ => Pass1, GPT disk, loading from EFI - OK

[Azimutz]

Ok, i found my previous notes from testing boot0work (thought they were lost);
i'm testing again and the results so far match the previous ones, which are not exactly as i recalled.

In case someone read my previous post (replaced by this one), i'll post it again, this time
with fresh info

[dmazar]

I just came back here to write a reply to your previous post, and found out that I do not need to do that. Well,  this saves me some minutes

[Azimutz]

Ok, glad i posted the warning
I have most of the testing done. Checking now against your results; i have some conflicting results so,
i'm repeating some of the test cases, in case i messed up something or didn't noticed other/s.
I'll post results later today; this time will not stop until this is done

[Azimutz]

Hi Dmazar, here i am... sorry for the delay, was caught in traffic

So, at the time of this post

Ok, i'm on to it in fact, i already have; been...

Will post details later... so far so good, looks promising.

i had done a ton of testing with boo0work, but the notes i had taken where lost and i was posting from memory;
after finding the notes, i realized that my memory was too optimistic regarding the Hibernation feature.
Since i was going to repeat the tests, i decided to delete the "too hib optimistic" post, which i'm now editing.

Straight to the point, my test scheme is a bit diff, but confirms your results;
first my device layout:

Code: [Select]

/dev/disk0
    #:                       TYPE NAME                    SIZE       IDENTIFIER
    0:     FDisk_partition_scheme                        *320.1 GB   disk0
    1:               Windows_NTFS Windows 7               64.4 GB    disk0s1 <- active
    2:                      Linux                         32.2 GB    disk0s2
    3:                  Apple_HFS empty                   17.2 GB    disk0s3 <- test, OS X installer, etc...
    4:                  Apple_HFS Hyperspace              206.3 GB   disk0s4 <- data/boot1h
 /dev/disk1
    #:                       TYPE NAME                    SIZE       IDENTIFIER
    0:      GUID_partition_scheme                        *500.1 GB   disk1
    1:                        EFI                         209.7 MB   disk1s1
    2:                  Apple_HFS Snow                    100.0 GB   disk1s2 <- main system/boot1h
    3:                  Apple_HFS Home                    200.0 GB   disk1s3
    4:                  Apple_HFS Leo                     25.0 GB    disk1s4
    5:                  Apple_HFS Tiger                   25.0 GB    disk1s5
    6:                  Apple_HFS Lab                     25.0 GB    disk1s6
 /dev/disk2
    #:                       TYPE NAME                    SIZE       IDENTIFIER
    0:     FDisk_partition_scheme                        *2.0 GB     disk2
    1:                 DOS_FAT_32 PAPPS                   1.8 GB     disk2s1
    2:                  Apple_HFS Chazi                   213.6 MB   disk2s2 <- activedisk2 is my usual rescue usb stick, others are the local disks.
Disk0 is the first bios device, but disk1 is usually the first device selected on bios.
They all have boot0work installed (now V2) since those first tests, meaning that i've been using it
on a day to day basis, which is already a good test result

Booting from disk0:

active = win

-----//-----
usual setup
stage1 = d0s4, d1s2, d2s2 Finds: d0s4 - hib Ok
PD pt pt pt pt
PD pt pt pt L pt L

-----//-----
finds stage1 on the other disks?
stage1 = none, d1s2, none/d2s2 Finds: d1s2 - hib Ok
PD pt pt pt pt
PD pt pt pt L pt L
  D pt ppp G L L

stage1 = none, none, d2s2 Finds: d2s2 - hib Ok
PD pt pt pt pt
PD pt pt pt L pt L
  D pt ppp G L L L L L L
  D pt pt L

stage1 = none, none, none Finds: boots windows directly - hib Ok
PD pt pt pt pt
PD pt pt pt L pt L
  D pt ppp G L L L L L L
PD pt pt L

Booting from disk1:

active = doesn't apply on GPT

-----//-----
usual setup
stage1 = d0s4, d1s2, d2s2 Finds: d1s2 - hib No - resume/hib Ok
PD pt ppp G L L

-----//-----
finds stage1 on the other disks?
stage1 = d0s4, none, none/d2s2 Finds: d0s4 - hib No - resume/hib Ok
PD pt ppp G L L L L L L
PD pt ppp G L L L L L L
  D pt pt pt L pt L

stage1 = none, none, d2s2 Finds: d2s2 - hib No - resume/hib Ok
PD pt ppp G L L L L L L
PD pt ppp G L L L L L L
  D pt pt pt L pt L
  D pt pt L

stage1 = none, none, none Finds: boots windows directly - hib No - resume/hib Ok
PD pt ppp G L L L L L L
PD pt ppp G L L L L L L
  D pt pt pt L pt L
PD pt ppp G L L L L L L
  D pt L

Booting from disk2 (external USB):

active = s2

-----//-----
usual setup
stage1 = d0s4, d1s2, d2s2 Finds: d2s2 - hib No - resume/hib OK
PD pt pt L

-----//-----
finds stage1 on the other disks?
stage1 = d0s4, d1s2, none Finds: d0s4 - hib No - resume/hib OK
PD pt pt L pp
PD pt pt L pp
  D pt pt pt L pt L

stage1 = none, d1s2, none Finds: d1s2 - hib No - resume/hib OK
PD pt pt L pp
PD pt pt L pp
  D pt pt pt L pt L
  D pt ppp G L L

stage1 = none, none, none Finds: boots windows directly - hib No - resume/hib Ok
PD pt pt L pp
PD pt pt L pp
  D pt pt pt L pt L
  D pt ppp G L L L L L L
PD pt pt L pp
  D pt L

I did small variations from the above, that only confirmed these results (and others):
- active partition with boot1 is always found
- first hfs partition with boot1 is always used
- Windows always hibernates if it was launched by the boot0 on it's device

This time i'm not going to be so optimistic
Just going to say that from my point of view, i see no reason for not using boot0work as the only boot0.
The compromise between debug/verbose is acceptable, since the booter couldn't be compiled with both
enabled before. Any technical problems on the horizon?
There's already an issue filed on the forge; i recommended that we keep boot0 around, at least while we test,
but i think boot0hfs can be wiped out for good.



About bugs, going to mention this situation mostly to get your opinion and because i think it has to do with boot0!?

I came across this, one time i messed up the GPT table on disk1, and tested it a couple of times after that.
The outcome in this situation is that i'm no longer able to boot, even from the usb stick,
since the damaged table seems to be always checked and the boot just hangs (if i do recall...).
The obvious question is, can we make the booter fall back to one of the undamaged disks in a situation like this?

Just as info, the GPT table corruption was caused by Testdisk, while testing on Windows,
after a quick ride on OS X were it runs on read only mode by default, which seems not to be the case on Windows
The solution at the time was, use SuperGrub to load grub on disk0s2 and boot to both Linux or Windows to look for a
solution; it didn't took long to find gdisk, but it took hours of reading to use it to fix the table.
In the end, i managed to edit the table, all came out good and it turned into an educational experience
Needless to say that, these days i make sure everything is properly backed up on the usb stick, before i do such tests.
 
Ok, be back later...
And thanks for your work

[Blackosx]

Hi dmazar

Well done for finishing up here and posting your revised work. I've been busy with other stuff lately and haven't really had the time to run any thorough tests at my end, though I've been using the boot0work v2 for a little while now and have had no issues.

The tests both you and Azi have conducted show success in all areas. This is great and maybe this could become the one and only boot0?  I'll try to run some tests of my own this weekend.

[dmazar]

Hi Azimutz. Thanks for this. I'm not the only one with long hard-to-read posts here any more. 

I've only changed few lines in existing boot0 (99% is already existing code) and it took me more time for testing then actual coding. So I guess you also spent more time testing it then I did coding. Thanks.

There's already an issue filed on the forge

I did not understand this one. What's the issue here?

I'll write an answer on "About bugs" part later after checking the code.

This is great and maybe this could become the one and only boot0?

That was the plan. At least for me when I realized it could be done with just few tweaks. But my Hackintosh/OSX experience is limited to few months and I do not know if it covers all situations. I do not have experience with RAID, for example.

Regarding one and only boot0: somebody will have to add support for Advanced format 4K disks (not AF 512 emulation) soon, and who knows ... maybe this will require two separate boot0 again.

[Azimutz]

Hi Azimutz. Thanks for this. I'm not the only one with long hard-to-read posts here any more. 

yeah, i have a natural tendency for those

... So I guess you also spent more time testing it then I did coding. Thanks.

yep, your guess is right... i guess
You're welcome it's for a good cause. Sorry for taking so long to feedback...

I did not understand this one. What's the issue here?

aah... there's no real issue; in this case the issue on the forge is just a "todo".

I'll write an answer on "About bugs" part later after checking the code.

No hurries...

... I do not have experience with RAID, for example.

Regarding one and only boot0: somebody will have to add support for Advanced format 4K disks (not AF 512 emulation) soon, and who knows ... maybe this will require two separate boot0 again.

Yes, we do need some RAID testing (can't do it my self atm); i also didn't tested boot1f32 but will soon.
And you're probably right on the 4k disks... we'll see...

[dmazar]

About bugs, going to mention this situation mostly to get your opinion and because i think it has to do with boot0!?

I came across this, one time i messed up the GPT table on disk1, and tested it a couple of times after that.
The outcome in this situation is that i'm no longer able to boot, even from the usb stick,
since the damaged table seems to be always checked and the boot just hangs (if i do recall...).
The obvious question is, can we make the booter fall back to one of the undamaged disks in a situation like this?

As far as I can see, boot0work searches/checks for valid HFS+ partition type (in MBR and GPT), for GPT signature and boot signature (for boot0 and boot1) before loading and jumping to boot1.

In case of disk corruption, proper way would be to check GPT table CRC and then use backup GPT table, but this is not an option due to limited code size.

What happens currently is: if above checks are not present, boot0work will skip to the next disk. But in some cases it will just write boot0: error and stop. This will happen if partition types and/or GPT signature are ok, but some LBA (GPT array LBA or partition start LBA) are invalid for this disk (pointing to non existing blocks). And this will happen on my hard disk only - on my usb it gets over it and skips to another disk (I could not resist to test it).

This could lead to a non-booatable system in some cases. For example on my current setup:
disk0: windows with boot0work in MBR
disk1: GPT disk with OSX
If disk1 GPT gets corrupted in a way that boot0 stops with boot0: error, then I can not boot windows also any more (phase1: hfs+ not found on disk0, phase2: stops on disk1 and never gets to phase 3 to load windows from disk0 as active partition).
My options then would be to disconnect disk1 (boot0work would load windows in that case), or use usb/cd/dvd to boot windows, linux or other to recover GPT disk. And then to spend 2 hours on trying to fix it with iBored, find out that it does not work properly for some reason and then to realize that gdisk for windows exists which is able to do the job. Excellent. Well ... this could all happen. Or happened. 

Anyway, this stopping of disk scanning can be resolved by changing two lines of the boot0work code:

line 533 in checkGPT: (old code commented with ; new added)
;jc       error
jc       .exit


and line 606 in loadBootSector
;jc      error
or   dl, dl ; to set flag Z=0
jc      .exit

I could not resist to test it, and it seems to work. But full testing requires going through all normal tests again plus corrupting GPT table in various ways and make more tests. That's not easy, and … hm, not very interested in doing it. It does not look so important to me, or …?

Hm, wanted to make small post, but it turned into large one again. Maybe next time ... 

[Azimutz]

Back
Dmazar, just came from testing the GPT corruption and it seems my memory may have done it again..!?
Well, i also understand this stuff a little better now and the debug build helps spotting the problem.
So, i patched boot0work as you instructed and installed it to the usb stick.
Then booted windows and with gdisk, backup + print + verify GPT:

Code: [Select]

Command (? for help): p
Disk /dev/disk1: 976773168 sectors, 465.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 68E983C6-1E81-4FC3-ACF1-3C8455A5413F
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 976773134
Partitions will be aligned on 8-sector boundaries
Total free space is 243941613 sectors (116.3 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              40          409639   200.0 MiB   EF00  EFI System
   2          409640       195722143   93.1 GiB    AF00  Snow
   3       195984288       586609287   186.3 GiB   AF00  Home
   4       586871432       635699559   23.3 GiB    AF00  Leo
   5       635961704       684789831   23.3 GiB    AF00  Tiger
   6       685051976       733880103   23.3 GiB    AF00  Lab

Command (? for help): v

No problems found. 243941613 free sectors (116.3 GiB) available in 6
segments, the largest of which is 242893031 (115.8 GiB) in size.Note that i have unpartitioned space after part 6.
Next to mess the GPT, used Testdisk to change EFI partition type to EFI, FAT32;
it's the closest i can get to the original circumstances and never fails the result:

Code: [Select]

Command (? for help): p
Disk \\.\physicaldrive1: 976773168 sectors, 465.8 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): CABED121-6A46-7F44-ACA5-6F75B0703CB3        <---
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 976773134
Partitions will be aligned on 8-sector boundaries
Total free space is 146285357 sectors (69.8 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              40          409639   200.0 MiB   EF00
   2          409640       195722143   93.1 GiB    AF00
   3       195984288       586609287   186.3 GiB   AF00
   4       586609288       635437415   23.3 GiB    AF00            <---
   5       635440902       684269029   23.3 GiB    AF00
   6       684527684       733355811   23.3 GiB    AF00
   7       733617956       782446083   23.3 GiB    AF00
   8       782970372       831798499   23.3 GiB    AF00

Command (? for help): v

Problem: The CRC for the backup GPT header is invalid. The backup GPT header
may be corrupt. Consider using the main GPT header to rebuild the backup GPT
header ('d' on the recovery & transformation menu). This report may be a false
alarm if you've already corrected other problems.

Caution: Partition 5 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Caution: Partition 6 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Caution: Partition 7 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Caution: Partition 8 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Consult http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/
for information on disk alignment.

Identified 1 problems!Like this i can't boot from any device.
It get's passed the boot0 debug output and hangs at the first spin or two of the cursor,
which means boot0 is not involved, right!? If so, sorry to make you waste time with this.

Thanks. See ya later...

[dmazar]

Like this i can't boot from any device.
It get's passed the boot0 debug output and hangs at the first spin or two of the cursor,
which means boot0 is not involved, right!?

Seems so. If you are booting from this drive, then I guess boot0 will find boot1 on 2nd partition and continue with booting, but since GPT is not valid, maybe Chameleon stage 3 gets confused. Anyway, I guess you would not be able to boot anything from this drive, but if the issue here is in the stage 3, then this could be fixed to allow to boot from some other drive.

boot0 fix from my previous post would deal with the  corruption of LBAs (GPT array LBA or partition start LBA), but only if those LBAs are out of the range for the current disk. This is just to avoid boot0 from stopping in case of errors and to allow it to continue with boot1 search on other partitons and disks.