Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - ~Poke~

Pages: [1] 2
1
Generation VI Glitch Discussion / Re: Invalid Pokémon in Gen 6
« on: January 07, 2016, 09:20:39 pm »
"3DS kernel hacking is a thing now. It's possible to downgrade firmwares 9.3 to 10.3 back to 9.2, and then, downgrade further to 4.2 and have custom firmware (rxTools) to pirate games) http://i.imgur.com/fIl8Jyf.jpg "

My friend just posted this on skype. It might be useful to some of us for continuing the Gen 6 Invalid Pokémon research, for those who were forced to upgrade firmware in order to play some of the new games. I don't actually know much about kernel hacking, myself, though.
It is indeed possible, though a little risky. The consensus seems to be it's mostly safe and definitely worth it for old3ds, but less safe (and less worthwhile) for new3ds. It really depends if you still want your 3ds to work as a normal console - with old3ds you can run a thing called emuNAND to be able to run the latest firmware from within the old exploitable one, so you get the best of both worlds. On new3ds the highest version you can run in emuNAND is 9.5, so you still need all sorts of other hacks to get into eshop, pokemon bank and etc. and need to pirate a cryptofixed version of any game using the 9.6+ cryptography.

I don't really think it's worth it for new3ds since being able to downgrade relies on kernel access - so in theory 10.3 is fully exploitable, just no one's done it yet. Maybe if 10.3 emuNAND is possible it would tempt me though.

It's also possible on latest firmware to pirate "legit" CIAs - basically, games that come preinstalled on special edition consoles, since they're not encrypted per device. This includes all the Pokemon games, so if anyone wants a spare game for glitching you can look into that.

(What are people using to edit the glitchmons into their party on 9.3+? is it just PKHex? I looked into a homebrew called PCHex to do it all on the 3ds but it doesn't allow glitchmons). Never mind, went back and read the first post :P
2
All of this is contingent on being actually able to keep those indoor npcs present for the duration of the whole run. I am unable to reproduce your different behavior for real maps vs MZs - I just tried from my hacked-to-HoO save and Maylene vanished both when I went west (Iron Island) as well as south (MZ). Interestingly, the glitch blocks from where the shovable doors in the Veilstone Gym were remained (and could be moved) :-\ .
There's still the trick of getting there without reloading the NPCs though, like you say. It's a shame the Mystery Zone thing didn't work in that situation.

That's an ingenious idea; can we tweak inside Amity Square (to prevent the lady from putting our pokemon back into its ball)?
The map looks like it's probably big enough? I can't find any mention of it on Google or Youtube though. I do think it was discussed or at least mentioned on the HoO forum, so maybe someone with that backup can do a quick search?
(I'm not good enough at tweaking to test... though maybe what we need is a lua script to tweak for us :P)

I don't see how the NPCs could be made to appear, given that you're on something like [30000,20000] (just making this up :P ), where no npcs have been programmed? Unless you want to S+R, but then your matrix changes to the HoO's (assuming you S+R inside the HoO), meaning you'd be surrounded by Iron Islands and Solaceon Ruins, none of which maps have usable npcs. (If you could make npcs appear though, you might be able to talk to them right across the map boundaries?)

This video shows it. (on real hardware!) Looks like maybe they use a 2 byte location, so they'll appear in Fake Jubilifes. Should work for any npcs provided you're in the right place and header, though that sounds really rare for anything other than a Jubilife (unless the indoor repetition lands on it? So diagonal Fake places?)
The video looks like they can't be talked to, but maybe that can be worked around? Might be a Z thing again, since Jubilife is raised.
I had some weird Z things happening at North Fake HoO so maybe that can be fixed somewhere in Fake Sinnoh.
3
The overworld is useless for us, since they won't contain the npcs from indoors we'll use to trigger the Arceus script.

My observation was the following: if I save+reset in a pokemon center and then change my coordinates using pokesav, the npcs from the pokemon center are still there at their original coordinates and can be talked to to trigger the Arceus script. This means that if we want to battle Arceus we must:
  • (save and reset inside an indoor area?)
  • find a way from there to the HoO without having to save and reset again (at all or just in the OW?)
(I really really hope this 'npcs are stored separately from map location and hence will still appear if you've underflowed x/y as long as you haven't entered a warp' holds though if we've walked there ourselves, rather than haxed it. If not, then we're screwed and won't be able to trigger the script after all  :-X )
Hm, the main problem with that I'm seeing is the NPCs have the specific map coordinates they're set to, so you would need to move the Hall of Origin to those coordinates. The only way to do that I know of is save and reset there? Maybe some kind of key item could do something? Another option is going to Fake (your place), which I think will only show the NPCs if you're in the right header (as happens with Jubilife in Fake Sinnoh) but I've never tested with an indoor area so perhaps it's different.

I do suspect that NPCs are reloaded just by travelling - you can ride all over the overworld without going through warps via walk anywhere codes and all the NPCs work fine. That may only be for the Overworld though, since indoor areas don't have much reason to dynamically load NPCs and there are some exceptions in the Overworld (I just looked. On Route 201 you can still see the head of a Twinleaf guy, but if there's no inbounds place to see a neighbouring NPC then you can see it unload when switching areas. One example is Canalave city.)

With a bit of quick testing, visiting a Mystery Zone from inside a building has no affect at all on the NPCs. Visiting a different header (in this case Floaroma Meadows) will make them invisible. Sometimes they can still be talked to (people in Poketch Co.) but sometimes they can't until the graphics are reloaded (Your mum at home). I don't really see what difference leads to this, but I notice your mum's shadow is in the wrong place so maybe it's a Z thing? Why do buildings have different Zs?
I hope this doesn't mean walking from the building to the HoO entirely in Mystery Zones, that sounds tedious :P

I feel like I might be on to something there, except for the slight detail of needing to be in the HoO for the right scripts to load and needing to not enter anything so the right events don't unload... Perhaps there's some distinction between scripts and events/NPCs that will let us abuse this?

What if the NPCs that follow you could be brought there? They're only meant to go to specific areas, so they probably call specific scripts that are within them, right? That does potentially mean starting a new game to get the one that runs the correct script, if the repeatable ones (only Amity Square off the top of my head?) don't do it.

You saved and reset at 430N, didn't you? I don't know why, but that seems to wipe the npcs from your save, meaning you won't have those npcs anymore to trigger the script...
(I do hope very much I'm wrong though, in which case you should be able to walk to [0,0] and see the npcs from Poketch Co, and if you're then still in a HoO - you should be given that you're in the HoO matrix - you can talk to two of those npcs to trigger the battle)
Yep, I'm using the standard 430N tactics. Even if this one isn't useful for catching Arceus, it's still cool to be able to travel there since it hasn't been found before.

Fake Jubilifes (including the one at 430N) are set to the same scripts, events and therefore NPCs as real Jubilife by the way. This doesn't really help since they're all the way over at the coordinates of Jubilife City.

Another idea: Void areas repeat. In Fake Sinnoh, if the visible Jubilife City overlaps with a Jubilife header the NPCs will appear (in theory this goes for any other city or route, though you're incredibly unlikely to find the header)
If there's an area where the Hall of Origin overlaps with Fake Jubilife City, well it turns out that the HoO I found is surrounded by Jubilifes so we can make the NPCs appear. They would disappear again on entering the HoO but maybe that can lead to something? *shrug*


Why don't you simply use 'find' (assuming Windows)? Open a command prompt, cd to the folder containing the script, enter command '<VoidScan.txt find "00510"'. If a blank line is returned, you haven't found a HoO yet. If a massive line is returned, you got it. If you have 'grep' available (not standard Windows) you can use '<VoidScan.txt grep -Po "00510\[[,0-9AF]\]"' to then print the exact coordinates.
That... sounds a lot more convenient. I'll do that next time files get too big :P

That script never went south and/or east, but I'll include those directions in a future iteration nonetheless.
The script you wrote to try travel to the SPHoO treated the coordinates as 2 byte, so when it underflowed it only went to 65k. So after seeming to travel 70k north, it had effectively traveled 60k South. You mentioned Victory roads being either there or on the way :P

Since the scan is still running, I just found another overworld void HoO. This is a different one rather than a repetition, because it's next to a Mystery Zone instead of Jubilifes.
00510[FFFFFC60,FFFD7E80]
Unless the neighbouring Mystery Zone is useful in some way I don't see any reason to actually travel out to this one :P
4
I've been thinking. The script in its current form takes ages to run with no certainty of success. However, maybe it's possible to cut down on the runtime by not checking every 32x32 area.

The doc says that the void repeats diagonally. Is this always true? If so, we could change the algorithm to:

Correct. For indoor areas the repetition is directly diagonal - 32 steps up and 32 east will lead you to the exact same area.
It's different for the overworld though. "Sinnoh requires 960 steps east before you reach the same area while indoor areas require 32."
If you have a way to run the big numbers and want to confirm this, I have a scan of the overworld that's up to Y: FFFFE000. Yeah, it's been running for a while (I have no clue how long, since it seems to pause when the screen turns off).
It sadly hasn't found a Hall of Origin yet (If areas do repeat at 32N 960E then I would expect to see my really far away one... Eventually? Apparently not yet.)

My preferred way to visualise this would have been with a spreadsheet, and fill in all the Mystery Zones and Fake Jubilifes with a dark grey. Then zooming out would make it easy to see any other maps.
As it stands, the only thing that'll open this without crashing is Notepad++ and even that's getting slow, so I have no idea how to go about visualising it. At least I can still search it.

The scan (47mb) is here. I'll stop this scan now since we're figuring out more efficient ways to do it :P
https://mega.nz/#!vphD2CYb!V-XbRrZUap0iIakg_-yGBAVLsp3FhCjo5WF4jHpJetw

Maybe next for the overworld I'll do (960/32 =)30 spaces West, infinitely North. That'll fit within a spreadsheet and should eliminate the repetition.

I was thinking maybe I was wrong to exclude SFS and EFS from consideration. My reasoning was that they would just be mirrors of real Sinnoh, but this is only the case if starting (a) from [0,0], and (b) from the overworld; on top of that, real Sinnoh is not 65536x65536 tiles in size so it might still hold some secret HoOs? Though on the other hand, their coords will obviously not underflow, so they might not load the improper maps from before the beginning of the real matrix data (I'm assuming that's what's happening) that we rely on?
There are more things down there, since your script that went to 65k had results. Even if they load data from after the matrix rather than before, that still sounds useful.

Wow, you two seem to be very tech-savvy about this. I feel a bit overwhelmed by your posts :-[ :)
Well we're nearly through making this script work, maybe we'll make a bit more sense after that :P
The other puzzle is how to use the Hall of Origin once we get there, and I don't have any ideas for that yet.


EDIT:
The Hall of Origin has been found!
00510[FFFFFF60,FFFF8720]
30,528 steps North and 180 steps West of 430N.
That's really close.
I just made the trip manually and it is there! Correct music and everything! (This also confirms that scanning with the script is accurate).
Now to check a fresh save file, to see if it's consistent.
5
see above - due to a mistake on my part this is actually 400/32*65536 further West than the log would have you believe - I'm so sorry  :-X :P

This was before any looping at all - it was still the version of the script that goes west until it crashes.
I'm looking at the outputs and seeing 00002[0000FFE0,0000FFE0] when the coordinates were actually [FFFFFFE0,FFFFFFE0]. None of them have the higher end set to anything but 0, so I think it's got the wrong address... but it's the same as the others? It's reading it wrong maybe? Are you getting this?

No, because I wanted to make it realistic for doing a real run later on, and if you enter a door from behind you're not at [0,0] but rather at [door.x,door.y+1]

Ah, that makes sense. For overworld I think I'll make it start at 430N from now on, instead of 0,0 (although, that does mean intentionally ignoring a Hall of Origin that I do know exists ~400 South and (something big) West of 430N... but hey there'll be plenty more where that came from.)

I'll just edit this reply in since I don't have anything new post worthy yet:
You mean it hadn't gone south at all yet?
No, I'm not. It's possible that it got the wrong address if you resetted without restarting the script (that would not update the mapdata_ptr value). That's the only explanation I can think of - everything seems fine on my end (I'm testing from behind the door inside the building on Iron Island, and a random excerpt of my log is '00000[FFFC71E4,FFFFFF69]').
Yep. Sorry for the panic though, the current script is working... I don't see any difference in the script for file writing so I can't say what the problem was? Sorry, but it works now. I've been stopping the script, restarting the game and deleting the .txt between attempts. It probably was my error though, more likely to be mistakes than just a random occurence :P
6
Oh no, it's just that I'm printing a map every 32 steps. So if a map is bigger than 32 steps it will be printed multiple times.
Usually if a map shows up it is only 32 steps across though - eg. Stickyman's HoO. This is only a 32x32 area in the void even though when the real map is visited it's 64x64. Sure it is possible for a map to show twice in a row, but in VoidScan.txt I don't see any single maps at all. Always 2, 3 or sometimes 4 of the same in a row.

I changed the script to wait 4 frames instead of 1 (just a guess, maybe less are fine) and I'm getting very few duplicates. Still some, but few enough that perhaps those duplicates are real? Of course this does slow the scanning down.

I've actually already found a HoO (it's 510 btw, not 516)
Nice! When I said 516 I was referring to the elevator that I suspect is causing the crash btw.

but I had sort of cheated while doing it... My starting point was the Floaroma Flower Shop, but before I began searching I warped myself to [0,0] on that map, hence the cheating bit. But with that starting point, I found a HoO somewhere in like WWWWWWWWFS or something.
Oh, does the script not set x and y to 0 before it starts? I guess it doesn't, I'll add that now. I've just been running it from the middle of Jubilife, oops. Well, I guess it doesn't really matter since it gives coordinates anyway.

I'm now seeing what happens if I limit the script to maximally five FSs in a row and then trying from various entry points. Because it's going to take aaaages to map all of the void, and just as many ages for a non-cheater to walk normally to WWWWWWWWWWWWWFS or something like that.
Yeah, limiting the search like that sounds good. Those 5x64k steps should give plenty of space for interesting things to show up.
Pawny mentioned earlier that the voids of many inside places are similar/less interesting, so the Overworld might give better results.


I just got another crash warp that sent me to map 113, matrix 210. That header is part of Hearthome, perhaps the department store so I guess I ran into the other elevator now. I got that when I started from 0,0 and didn't have someone to talk to, so all the more reason to use that sneaky trick which does seem to work.

EDIT: It found a Hall of Origin in the overworld void at 00510[0000D8E0,0000FFE0]
So that's about 55,520 steps West and 400 steps South of 430N.
No wait, no it's not. Ok the script is only writing the small end of the coordinate to the file it looks like? Oops.
7
Nice!
I'm getting an event that may be the same crash you're getting. The character walks 2 steps up, 2 steps down then goes through a warp.
While running the game at real time this causes a crash, but if I fast forward the warp works and I'm teleported to matrix 230 (apparently Vista Lighthouse). The script keeps running, but the results are now incorrect since it's the wrong matrix so it's still a problem.
Maybe if the script sets the matrix each time that would be a workaround, but since fast forward actually affects how the game behaves it might be safer to not use it, so we'd need to avoid this completely.

The problem area wouldn't be the lighthouse itself, but the area that warps you to it. The elevator at Map 516 Matrix 207 does this. The script is triggered the moment you enter, no interaction. The doc mentions this elevator and one in Hearthome causing problems under "I got kicked out of the void!"

This is an autorun event that can't be disabled, so the only way I see to completely avoid it is reverting to a saved state and skipping that area. Saving states on every loop of the script sounds slow, so you'd need to choose a reasonable interval and make sure the script can deal with being a little behind where it was up to.
That said, I may have found a sneakier solution... talk to someone. While their message is open, start the lua script. This seems to stop the warping script from running since you're busy. The textbox also bounces with the place name which is pretty silly to watch.

No guarantees it's a perfect solution. I'm up to FFF00000 so far without problems, but searching the output shows I haven't been to map 516 yet. That's a lot further than I got on other attempts so it might have solved something?

Just before hitting post my screen turned black at FFE?????. The script and game are still running, maps and music still changing, but this may be a limit for real exploration. I'll try to test it manually in another instance (which could take a while and/or be hard thanks to autowarps :P). VoidScan.txt still says I haven't hit map 516 so it's not related to the warp... still could be related to having a textbox open though.

EDIT: I just ran some numbers and that would take something like 3 hours 40 minutes to ride to, at the rate of 10 spaces per second (which probably isn't completely accurate). I think I'll pass for now. Meanwhile the script is still going well, now just passed FFCD0000. Still doesn't say it's found map 516 (neither map 511).

EDIT again: Just crashed at FFC444AC. The last 7 maps it visited were map 0, but before that was 38. The Canalave Library.
Something I'm noticing is that every map it visits is repeated. Here I see three map 38s in a row. I don't think the void normally acts like this though, maps tend to be alone (long stretches of mystery zone or the various jubilifes being the very notable exceptions that make it seem otherwise). Is the script running too fast for the game to keep up maybe?
8
There's no way to make the window bigger, AFAICS. My intention was for someone to copy this into notepad/excel and then examine it from there.
I've tried this, but it has a maximum amount of text. After too much is written the older data is deleted. So if it's possible to write a file that would be really useful. I also tried converting it to csv (find and replace spaces for commas with notepad) but Google docs didn't like it (no reason given) and libreoffice said it exceeded the maximum number of columns. So that's a shame. It also doesn't display properly in notepad with word wrap disabled, apparently it has a max line length.
Notepad++ shows it fine so that's one way to read it.

Looking at the first few lines, the script doesn't seem to give correct results though. When both x and y are very low in matrix 0 they should show the real layout of the overworld, right?

That's my point, at 65536 it should just have rolled around so 70k==4.5k. But there's a different map at 4.5k than at 70k, so something else must be going on.
I tried to test this, but at one point the player enters a warp and crashes. I wonder whether that's a problem with that spot, or it's because of how the script works. x:17792 y:5664 was where I got it, though idk if it's consistent.




I'm sorry, I thought I had implemented your suggestion of treating the coordinates as 4-byte data rather than 2-byte but it seems I hadn't.

Changing the words to dwords gives different results - the previous script now puts me squarely at Route 205. This is still not the promised SFHoO, but it's different compared to the Mystery Zone (which becomes Oreburgh City when you save in it) that I got when I was reading and writing words.



EDIT: Also tried this, but this BSODs me and then dumps me at the Sunyshore elevator right before it finishes walking 2912E after the 70kN

EDIT2: actually, this is wrong, because it's incrementing by incr*dir whereas it should be incrementing by just dir. But this dumps me into a MZ when I'm supposed to SR after 70kN, 2912E so that isn't right either.

I walked manually to the SPHoO to test if it was right, and it did show up. The top left coordinates are (B60,FFFEECE1) if that helps you, bottom right being (B7F,FFFEED00).
Also, apparently the script gets confused if I try to output the 4 byte x or y as decimal while they're negative? They get stuck at -2147483648 and won't change unless I make it positive/overflow to 0. There's probably something I'm misunderstanding but I blame lua.

While testing that I found something weird just to the west of real Sinnoh (when x underflows). It's basically... the opposite of Fake Sinnoh? No graphics load, but movement permissions still exist and you can find wild Pokemon.
This is really quick to reach so it probably is already known, but I can't find any videos about it. I think it should be called Invisible Sinnoh :P

[youtube]https://www.youtube.com/watch?v=loahWwUxymA[/youtube]

This video starts from 430N. I couldn't find any wild Pokemon without first turning on walk anywhere to get through the walls, so maybe no one ever realised how this place acts. Since some things are intact, I wonder if scripts load and etc... though no npcs are visible so they probably can't be interacted with if they are there.
9
What it's doing right now (or at least, it should; I haven't tested it all the way) is printing a map to desume's stdout (the lua window). That it, for every 32 steps it takes to the right, it prints a zero-padded map id plus a space, and once it's reached x=65536 (which is the same as 0), it increments y by 32 and starts a new line. As such, when you let the script run, you will have a full map, and if there's a 510 in there all you can immediately see how you get to it.
It's just a little hard to look at is all, unless there's a way to get a horizontal scrollbar? (Assuming the new line of void prints on a new line? Yeah I guess it does.)

Well, we know that all of FS is basically s**t due to not being loaded properly (e.g. lots of routes missing or not lining up correctly), but for our purposes, none of this is relevant - the only thing that matters is that we can walk anywhere (barring z issues) and that we have to find a way to reach a map that is of id 510. (And then hope that it's still going to be a 510 after we walk back to whereever our script #1/#2 npc is.)
All the maps are there, minus tilesets which makes them look weird. I guess since neither headers nor even collision dara are there it's more "Fake" than "Real", but my worry is that if the coordinates really are 2 bytes then they can't distinguish between Real Sinnoh and any of the fakes.

Do we know what happens of you go to Fake Sinnoh then back to Real Sinnoh? I'll go test that now.

Well, something must definitely be going on, since the FS at 70k N should be exactly the same as the FS at 5k-430 N otherwise, and this is evidently not the case. Does my script correctly reach the Stickyman HoO? If so, then data changes along the way, but my script manages to emulate that properly. If not, then either data does change along the way but my script's way of "walking" doesn't trigger that (so we'd need to reprogram it to also do stuff to these other RAM pointers you mention), or the data doesn't change but these RAM areas do change as a function of real walking and we will still need to do the same.
Your script only increments to 65k, since that's a full 2 byte number so I don't see how it could reach 70k. I haven't run it extensively though, and in theory (so if it does wrap after 2 bytes) that'd show up near the end of your script since 70k North would be around 60k South, and you do horizontal first.
I don't really get how both a 2 byte and 4 byte number can be mapped to the same thing which sounds like an issue (unless you've just got the small end of it?)
Maybe if you just change both it'll work.


Quote from: doc
As a footnote I might add that going too far in either direction will lead to all areas acting like BSoD
This is something to keep in mind. 4 bytes gives us 4,294,967,295 possible values for each coordinate which is way too many. That takes too long to measure, too long to travel to manually and the game isn't stable enough it sounds like, so the search should probably be restricted... though hey if your method of virtually traveling doesn't suffer from instability maybe it'd be interesting to measure anyway? It'll take forever though.
10
Nice work on your script! I tried it out myself and it does look rather funny :P
By the way, I think it does show valid data as explained below, so maybe all that's needed now is an organised way to output it. Perhaps just .csv (basically plain text) so we can load it into google docs or excel or etc. Another approach would be to ignore most values and have it search for some specific ones, outputting their coordinates. I guess it depends whether we want a hard to look at map of the entire void or a way to search for specific places, both of which may differ between save files.


Note: this code still suffers a critical defect for being valid for void exploring: when walking through the void, the matrix sometimes changes. This will affect what map ids you will get next. However, since this script does not actually walk, these matrix changes are never triggered. Thus, once you cross a matrix boundary, the results from this script would no longer be valid. I don't have the knowledge to fix this - we'd need to know what exactly triggers a matrix change, and debugging this is just miles beyond me, I'm afraid.

I don't think the matrix does change while exploring. I just did a short test, see this video (A little over double speed because I wanted to test quickly).

[youtube]https://www.youtube.com/watch?v=UVuOdV1REWk&feature=youtu.be[/youtube]

Basically when going through warps or saving/loading the matrix depends on the header. When travelling between areas as you do on the overworld, in the great marsh or in the void (of any matrix), headers depend on matrix.
I'm on a trip to Fake Sinnoh now to confirm this, since this was just a short journey.
Edit: I got to North Fake Sinnoh. Matrix stayed 0 the whole way, but... I can't believe I forgot this? The headers don't line up with the world? So I guess there is something else to how they're loaded... :/
Apparently I need to go look over all the basics again haha. Saving in NFS should put me there in the real world right? Anyway, this script is a good start.
Quote from: doc
Stickyman’s SPHoO void: 70,000 North, 2,912 East
Maybe some coordinates are stored as 4 byte rather than 2 byte? I hope it isn't just that along the journey the good data is changed (though that sounds very possible) because if it's path specific I don't see any trustworthy way of faking it like this, we'd have to make the lua actually travel it properly. That would be so much slower and also means we don't avoid the Pt/HG/SS crashes that this script probably would? (not tested).

Edit again: Yes, I'm seeing three 4 byte x coordinates. They do underflow to full 4 byte numbers, so these must be why the headers are different.
They're at mapdata_ptr + 14AC,  24AE4,  24AF0

Here's the script I used, a slight edit on your first one:
Code: [Select]
local function whereami()
local aslr_data_ptr = memory.readdword(0x021C4D28)
local mapdata_ptr = memory.readdword(aslr_data_ptr + 0x4)
local map = memory.readword(mapdata_ptr + 0x14A4)
local x = memory.readword(mapdata_ptr + 0x14A4 + 8)
local y = memory.readword(mapdata_ptr + 0x14A4 + 12)
local mat = memory.readword(mapdata_ptr + 0x00022B2E)
gui.text(2,3,string.format("Map: %d, X: %d, Y: %d, Mat: %d",map,x,y,mat))
gui.text(2,182,string.format("ptr: %x",mapdata_ptr))
end
gui.register(whereami)
It doesn't show fps itself, I just turned that on in the HUD.
11
What is interesting is that it seems to warp correctly now, but that it still fails to load the next map on its own. That is, if you uncomment the line 'force(0,8,21,65505,0) -- one step before SS', your new location will be correctly set to a few steps above Poketch Co such that, if you take one more step, you will enter the Floaroma Meadows. However, forcing the game to take that step by uncommenting the line after that does not do anything. Interestingly, you can't take that step yourself either - you're blocked by an invisible wall. If you save and reset, however, you can take that next step all of a sudden, and you will load Floaroma Meadows, but if you try to have the step walked automatically by uncommenting the _force(y,65504) line, it won't load Floaroma Meadows again. Unless, and this is the only scenario in which this works, you save one step before SS, load your save, and while at the journal pages activating the _force(y.. line; only in this case, you are correctly warped into Floaroma Meadows.
Nice progress!
On your previous script I was able to walk after warping with a walk anywhere code, try that.
12
The Hall of Origin loads script file 232, which is:
Code: [Select]
Script #1

CheckFlag 0x8E
CompareLastResultJump 0x1 Function_#1
End


Script #2

LockAll
SetVar 0x4118 0x0
Call Function_#2
Call Function_#3
PlayCry 0x1ED 0x0
WaitCry
Call Function_#8
PlayCry 0x1ED 0x0
Message 0x0
CloseMessageOnKeyPress
ClearFlag 0x8E
WildBattle2 0x1ED 0x50
SetFlag 0x8E
CheckLost 0x800C
If 0x800C 0x0
CompareLastResultJump 0x1 Function_#9
CheckWildBattle2 0x800C
If 0x800C 0x1
CompareLastResultJump 0x1 Function_#10
ClearFlag 0x11E
ReleaseAll
End


Script #3

End


Script #4

UseScript_#2
There are also functions and movements as part of this script file, but they're less telling so I'll leave them out.

As you can see script 2 involves playing a cry and starting a battle, this would be the Arceus fight. All script 4 does is call script 2, which is nice since it doubles our chances of making this work.
In the event viewer, the HoO has 2 events. One overworld (Arceus) who is set to script 0 and one trigger set to script 2. The trigger has a flag associated - this probably means it only appears when that flag is set, that flag being the event? It's 16664 if anyone knows.

Is this for Diamond and Pearl or Platinum? In Diamond and Pearl, the Arceus sprite will appear even if you don't have the Azure Flute (regardless of the state of the event flag) but this is not the case in Platinum, in which the Azure Flute is required to make the sprite appear.

That was Pearl. Yes, Arceus is always visible but does not have a script. The trigger (3x1 space that triggers a script when you stand in it) only exists when flag 16664 is set (0x4118 in hex).

In Platinum it all looks the same except that Arceus only appears when flag 590 (hex 0x24E) is set, and script 3 (empty in Pearl) changes some flags.
Code: [Select]
Script #3

028B 0x2 0x4000
If 0x4000 0x0
CompareLastResultJump 0x1 Function_#12
CheckFlag 0x11E
CompareLastResultJump 0x1 Function_#12
SetFlag 0x24E
End


Function #12

ClearFlag 0x24E
End

This script manages whether Arceus will appear, based on some other flags. Probably to make it appear on entering? Not completely sure without going through everything but that sounds like a safe bet. Scripts 2 and 4 are exactly the same except for the after battle checks, so calling them through another npc if possible will work just as well on Platinum as it would on Diamond/Pearl.

After battle in Pearl:
If you lost: go to Pokecenter & set flag 0x26C
If "WildBattle2" set flag 0x26C and say "ARCEUS disappeared from sight..." (is this if ran away? or defeated it?)
Else: clear flag 0x11E

After battle in Platinum:
If you lost: go to Pokecenter & set flag 0x24E
If "WildBattle2" set flag 0x24E and say the same
If 0x4056 = 0: set variable 0x4056 = 1 and clear flag 0x11E
Else: clear flag 0x11E

Another result to the battle was added. Is it for caught? Is Arceus not rebattleable in Diamond/Pearl? Or maybe it had to be reworked for some other reason. Either way it shouldn't cause problems.

Yeah. It's thanks to that we'd be able to enter Hall of Origin without walk-through-walls cheat in case we found it in the void. I don't remember if this is the exact coordinates but going something like 65536 steps east, 65536 north and then 65536 east again you should enter a Fake HoO in an HoO area, assuming you're starting from its initial area (the steps would vary according from where you'd start from).
Huh, did people ever try tweaking in the void to get back inside? Maybe that's too unreliable/unstable.

If anyone can help debug why this doesn't work, it would be greatly appreciated  O_o

Code: [Select]
function whereami()
aslr_data_ptr = memory.readdword(0x021C4D28)
mapdata_ptr = memory.readdword(aslr_data_ptr + 0x4)
m = memory.readword(mapdata_ptr + 0x14A4)
x = memory.readword(mapdata_ptr + 0x14A4 + 8)
y = memory.readword(mapdata_ptr + 0x14A4 + 12)
z = memory.readword(mapdata_ptr + 0x14A4 + 0x6C99A)

all_x = {0x226E7AC, 0x2291DE4, 0x2291DF0, 0x2291DFE,0x2333286,0x2333292,0x23332BA,0x2333E0A4}
all_y = {0x226E7AC+4, 0x2291DE4+4, 0x2291DF0+4, 0x2291DFE+4,0x2333286+4,0x2333292+4,0x23332BA+4,0x2333E0A4+4}
all_z = {0x21CEF76,0x2291E02,0x22DB13E,0x23067A6,0x232460A,0x2333296,0x23332BE}
for i = 1,7 do
-- The values above were taken from my own save state of Pearl. As such, we need to subtract my own map_data_ptr from them, and add the current session's:
-- (skipping over the eighth one since this only appears for x (and consequently y) but not z; its lack of a clear effect suggests the eighth value to be a false positive)
all_x[i] = all_x[i] - 0x226D300 + mapdata_ptr
all_y[i] = all_y[i] - 0x226D300 + mapdata_ptr
all_z[i] = all_z[i] - 0x226D300 + mapdata_ptr
end

all_m = {0x226E7A4,0x2291EC0,0x2291FE8,0x22ABCB0}
for i = 1,4 do
-- If we modify m (= all_m[1]), the game freezes. Perhaps we also need to update the others? Note interesting change counts: walking in and out of a building twice yields the expected change count of 4 for items 1 and 4, but 8 for item 3 and 10 for item 2.
all_m[i] = all_m[i] - 0x226D300 + mapdata_ptr
end

--memory.writeword(all_x[1],force_x) -- if this one is different from the others, the save file will be corrupted; if it is forced during save loading, the game will immediately crash when the save is loaded. This seems to actually move you collision-wise, but not visually.
--memory.writeword(all_x[2],force_x) -- the same; also if this one is forced, the save load screen will display a glitched white block. Does nothing live - perhaps this is the saved x position only?
--memory.writeword(all_x[3],force_x) -- changes map ID!!! but only live, not when loading save. Can be used to save inside a MZ (open start menu, change value to warp you into MZ), with bizarre results.
--memory.writeword(all_x[4],force_x) -- immediately snaps you into place, but without changing map id. Is saved when you save the game only if forced DURING the save, otherwise your new position is NOT saved. Interestingly, THIS IS ONLY GRAPHICAL
--memory.writeword(all_x[5],force_x) -- seems related to graphics loading
--memory.writeword(all_x[6],force_x) -- this one...
--memory.writeword(all_x[7],force_x) -- ...and this one are related; if one of them is flipped, the screen goes black and the other one starts counting like a madman. Counting resets when reloading the graphics. When forcing these to be set and taking a step, the perspective twists, showing this is contorlling the camera.
--memory.writeword(all_x[8],force_x) -- unknown?

function force_m(pos)
memory.writeword(all_m[1],pos) -- name tag. but based on real x,y,z! (i.e. OW locations get their name based on your coordinates)
memory.writeword(all_m[2],pos) -- graphics, but only after reload
--memory.writeword(all_m[3],pos)
--memory.writeword(all_m[4],pos)
end
function force_x(pos)
memory.writeword(all_x[1],pos) -- collision
memory.writeword(all_x[3],pos) -- graphics
memory.writeword(all_x[4],pos) -- matrix
--memory.writeword(all_x[2],pos)
--memory.writeword(all_x[5],pos)
--memory.writeword(all_x[6],pos)
--memory.writeword(all_x[7],pos)
end
function force_y(pos)
memory.writeword(all_y[1],pos) -- collision
memory.writeword(all_y[3],pos) -- graphics
memory.writeword(all_y[4],pos) -- matrix
--memory.writeword(all_x[2],pos)
--memory.writeword(all_x[5],pos)
--memory.writeword(all_x[6],pos)
--memory.writeword(all_x[7],pos)
end
function force_z(pos)
memory.writeword(all_z[1],pos) -- collision
memory.writeword(all_z[3],pos) -- graphics
memory.writeword(all_z[4],pos) -- matrix
--memory.writeword(all_x[2],pos)
--memory.writeword(all_x[5],pos)
--memory.writeword(all_x[6],pos)
--memory.writeword(all_x[7],pos)
end

force_m(3) -- Jubilife city
force_x(180) -- Jubilife city
force_y(777) -- Jubilife city
force_z(2) -- Jubilife city

gui.text(0,144,string.format("ASLR data: 0x%X / map data: 0x%X",aslr_data_ptr,mapdata_ptr))
gui.text(1,154,string.format("Matrix: %d, X: %d, Y: %d, Z: %d",m,x,y,z))
gui.text(0,164,string.format("X: %d %d %d %d %d %d %d %d",memory.readword(all_x[1]),memory.readword(all_x[2]),memory.readword(all_x[3]),memory.readword(all_x[4]),memory.readword(all_x[5]),memory.readword(all_x[6]),memory.readword(all_x[7]),memory.readword(all_x[8])))
gui.text(0,174,string.format("Addrs: %X %X %X %X\n       %X %X %X %X",all_x[1],all_x[2],all_x[3],all_x[4],all_x[5],all_x[6],all_x[7],all_x[8]))
end
gui.register(whereami)

Interesting script!
From just initial testing, I think there's some confusion between map and matrix. Eg. Jubilife city is map header 3 in matrix 0. Route 204 is map header 345 in matrix 0. I'm not completely sure what the script is doing, but if I run it indoors I end up in that building's void so the matrix isn't being set.

I think the map header should be considered a result rather than something to set - it's the place you end up being in at certain coordinates within a certain matrix. (It gets a little more confusing with being indoors since warps point at a header which then loads the correct matrix, but I think that's just because they decided it should work like that? That seems to be backwards from how the engine runs. I think the fact that other headers are found by walking out of bounds in that target matrix shows that something doesn't line up there. Or maybe not backwards, but still a way to call them in the other order. Maybe you could do it by setting the map if you use a warp at the same time.)

If I run the script while already in Jubilife it seems to line my X up with the pokecenter, but not Y so I guess that address might not be right, at least for one of them? Also graphical glitches but that might be because of the Y. If I run the script in a different route I still go to the X of the Jubilife Pokecenter door. If I walk around with the script running (only possible with walk anywhere, even if it puts me in a clear spot) npcs get cloned.

To search for the matrix address you'll need to find the find the matrix values with Spiky's DS Map Editor, since there's no real correlation between header number and matrix number.

(Interestingly, a lot of the generic buildings are different headers that call the same matrix. This means the exact same map (graphics and movement) is loaded but with different npcs, warps, text, scripts and etc.)

EDIT: Ok that difference between warping via header and travelling via matrix has got me thinking. The 3 layer matrices and the 1 layer matrices are laid out differently. The overworld/marsh are [headers, mystery zone heights, map files] while indoor matrices are just [map files] because the entire indoor place uses the same header. When you move between two matrix cells in the overworld it loads the map header, mystery zone height, and map data from the matrix in that order. Does that mean that it's trying to read 2 layers above the map data in an indoor matrix when you walk out of bounds? Or maybe even the same layer, since it sounds kinda illogical to reference things relative to the third layer... That might explain why indoor matrices are all so similar, if they're getting some data from the start of the map file that's mostly the same. That doesn't explain being affected by flags though... unless hey. If it tries to look "2 layers above" the map file in memory without putting any map header data in that memory, it'll be reading whatever's there just before the map data. That sounds possible?

This wild speculation is convincing me that the Great Marsh is going to be interesting. I just took a quick look now and riding North 800 steps from the top left has given me a couple Turnback Caves and Mystery Zones... but no Floaroma or Veilstone Store... so no second section?
13
3 hours after I started, I'm done writing this post. Sorry for the wall of text, I got carried away!
To clarify, when I said real/fake HoO earlier I meant 510/511. That was a silly choice of words when real/fake Sinnoh are a thing, sorry :P

EDIT: Figured something out. Walk into the Jubilife pokemon center, save your game, then use pokesav to change your map coordinates to 510,6,9,0. When you reload, you'll be in the HoO but with the Jubilife pokemon center's NPCs. Talk to the little girl in front of you to trigger a battle with Arceus. Now change your coordinates to 510,13,5,0. You're in front of a different person now, yet talking to him will also trigger Arceus.

This means that more than just one single script can trigger Arceus (but not all of them, e.g. Nurse Joy doesn't say 'Dodogyuuun' now :P ). Given that I just demonstrated that you keep the NPCs from the last map you save in, we will for sure trigger a battle with Arceus if, starting from a void area X, we manage to - without saving and resetting - find a HoO void when our x and y positions are at values that put us next to NPCs on our original map X.

Hey, I can explain that! Let's look at Spiky's DS Map Editor.


The Hall of Origin loads script file 232, which is:
Code: [Select]
Script #1

CheckFlag 0x8E
CompareLastResultJump 0x1 Function_#1
End


Script #2

LockAll
SetVar 0x4118 0x0
Call Function_#2
Call Function_#3
PlayCry 0x1ED 0x0
WaitCry
Call Function_#8
PlayCry 0x1ED 0x0
Message 0x0
CloseMessageOnKeyPress
ClearFlag 0x8E
WildBattle2 0x1ED 0x50
SetFlag 0x8E
CheckLost 0x800C
If 0x800C 0x0
CompareLastResultJump 0x1 Function_#9
CheckWildBattle2 0x800C
If 0x800C 0x1
CompareLastResultJump 0x1 Function_#10
ClearFlag 0x11E
ReleaseAll
End


Script #3

End


Script #4

UseScript_#2
There are also functions and movements as part of this script file, but they're less telling so I'll leave them out.

As you can see script 2 involves playing a cry and starting a battle, this would be the Arceus fight. All script 4 does is call script 2, which is nice since it doubles our chances of making this work.
In the event viewer, the HoO has 2 events. One overworld (Arceus) who is set to script 0 and one trigger set to script 2. The trigger has a flag associated - this probably means it only appears when that flag is set, that flag being the event? It's 16664 if anyone knows.

The 511 HoO doesn't have events, but it does have a script file. It turns out this is shared with a lot of Turnback Cave maps:
Code: [Select]
Script #1

End
Just a dummy. This is also set for EVERYWHERE and NOTHING Mystery Zones.

The level scripts sound important too, since in the GBA games they're the scripts that run automatically/on entering the area. But no matter which one I try to look at, it just contains:
Code: [Select]
Level Scripts FileMaybe I haven't checked a map that has them? That's what the restaurant has though, and it definitely kicks people out automatically. That's also what the Underground has and we know it can crash people on entering so I guess they're in a different format that SDSME can't read?

This functionality could probably be replicated with a trigger that covers the whole map, but the restaurant doesn't have any triggers. I don't think that would affect the void either, since the trigger has a specific X and Y location + area.

If only we knew where the 'next map id' bits were stored in memory, we might be able to write a lua script that determines whether the HoO is reachable from any given position...
A tip I can give you about that is that it's based on the map matrix. Matrix 0 is the overworld, It looks like this:

The selected place is at (0,0) so that's what 430N is north of, after resetting. If we can find where this is stored maybe we can see what incorrect data is loaded by incorrect coordinates? Alternately we could brute force it with a script that changes our X and/or Y by 32 at a time and records the results. (Maybe 31 at a time, then taking one step to make sure it's loading things correctly?).
(One note, I don't think the x and y that the new script fetches are settable? I'm not too experienced with RAM so I might be doing something wrong. The simplest way to do it manually seemed to be set a cheat then disable it, but the value reverts. Is this just a RAM mirror?)

Something else notable is that the overworld matrix has 3 "layers". One for headers (pictured), one for map height (only set for Mystery Zones around actual maps) and one for map files. We get a lot of junk data for headers but little or none for the rest? The "blackouts" referred to in the doc might be related to map height. The only map files seem to be Fake Sinnoh, when the coordinates loop. Perhaps that's handled differently.
Most other matrices only have a map file layer, so the headers must come from somewhere else. The Great Marsh has all 3 layers though, to allow different encounters in different parts. The matrix itself is much smaller, but perhaps it's void is as interesting as the overworld because of this? I'll have to explore this. There are no other 3 layer matrices in Pearl or Platinum.

HG/SS are also really interesting in that regard, since they have two 3 layer matrices being the overworld and the safari zone. That safari zone can be customised.

When you mean "real place", are you referring to an actual location or a void area with the same map ID as said location?

A HoO void next to (or 1 matrix space/32 tiles away from, or other distances useful in tweaking?) the loaded building that is the entry point. Thinking further on this, I don't think tweaking affects the map like that though. It moves/clears the terrain and movement permissions which are part of the map file itself, rather than scripts and events which are part of the map header. The only useful thing that stands out to me there is map re-entry without a walk anywhere code.

Here's yet another real Hall of Origin void which may or may not be the same one I found earlier. I only went west and south of 319W this time around.

The areas in the void for repeat to the North-East and South-West, or directly West where X underflows, North where Y underflows. The size of the matrix... might affect this? I kind of thought it did but now I'm not sure. The Poketch matrix is 1x1, and it's void looks like this in my save (allcaps is the Poketch that I entered from, with graphics and etc.):

Code: [Select]
[Mystery Z][Veilstone][Veilstone][Floaroma ]
[Veilstone][Floaroma ][Floaroma ][Poketch  ]
[Floaroma ][Poketch  ][Poketch  ][Mystery Z]
[Floaroma ][Poketch  ][POKETCH  ][Mystery Z]
[Poketch  ][Mystery Z][Mystery Z][Mystery Z]

Every [] is one cell of the matrix, and is 32x32 tiles.
The repeat on the underflow makes a bit of a mess of it, since you can see the diagonal pattern doesn't include the real Poketch Co.

Hall of Origin is a 2x2 matrix, and I'm seeing something like this?:

Code: [Select]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . [Iron  Island][SolaceonRuin][SolaceonRuin][Iron  Island]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . [Iron  Island][HallOfOrigin][HallOfOrigin][HallOfOrigin]
[Mystery Zone][Mystery Zone][SolaceonRuin][Iron  Island][HallOfOrigin][HallOfOrigin][HALLOFORIGIN][HALLOFORIGIN]
[SolaceonRuin][Iron  Island][HallOfOrigin][HallOfOrigin][HallOfOrigin][HallOfOrigin][HALLOFORIGIN][HALLOFORIGIN]
[Iron  Island][HallOfOrigin][HallOfOrigin][HallOfOrigin]
[HallOfOrigin][HallOfOrigin][Mystery Zone]
That looks like a mess so it probably isn't accurate. I'm not sure whether it's inconsistent depending on path, or I'm just losing track of it but I need to stop trying to count this out manually. The point is there's repetition, hopefully some lua can keep track of the specifics later.

I think this is may be because of the out of bounds coordinates on the matrix, eg:
[A1][B1][C1]
[A2][B2][C2]
[A3][B3][C3]
being stored as:
[A1][B1][C1][A2][B2][C2][A3][B3][C3]
Since the game knows the dimensions of the matrix, it knows where to read from. If you're outside that, everything breaks.

Either because it's so big, or because it's read differently, the overworld matrix doesn't seem to do this.

This post has been something like 4 hours to write so I'm going to stop it here. Sorry it's so big, there was so much to reply to then I got carried away testing things.
14
This means that Arceus might still be possible to find in the Void! If we happen to stumble upon an invalid map ID that the game considers to be HoO (just as many invalid OW map IDs are considered Jubilifes) in the void, and if this map contains at least one script (e.g. an NPC), then we can trigger a battle with Arceus!
This doesn't really add up to me. To have the Arceus script loaded, you need to be in the real HoO. To run the script, you need an event (npc or etc) set to run the script.

The effects you're getting sound like they're from being "In" the real HoO (at least partially loading it) but having the rest of the data loaded from another map.

I'm not sure if that can be done through the void. The only scripts I've seen run are the ones that start immediately on entering the map (Hall of Fame, Cynthia battle, getting kicked out of the restaurant at night, receiving the Pal Pad) because while in the void there aren't any npcs/signs/etc to trigger other scripts. Once you save, backtrack and enter the map you're actually in that map with npc data and all, so there's no different person to talk to to trigger the script.

That said, I'd love to be proven wrong. Perhaps if a Hall of Origin void can be found directly next to a real place, a tweak could be used to displace it, so you can be "In" it and talk to an npc from that real place. I don't like the odds of that showing up though, and I'm not sure if any tweaks really work like that.

EDIT: well, that was easy... here's a lua script showing your current map id and x,y,z coordinates (PEARL ONLY, probably also works on Diamond? probably won't work on PT/HG/SS):
Code: [Select]
local function whereami()
local aslr_data_ptr = memory.readdword(0x021C4D28)
local mapdata_ptr = memory.readdword(aslr_data_ptr + 0x4)
local m = memory.readword(mapdata_ptr + 0x14A4)
local x = memory.readword(mapdata_ptr + 0x2866)
local y = memory.readword(mapdata_ptr + 0x286A)
local z = memory.readword(mapdata_ptr + 0x286E)
gui.text(1,184,string.format("Matrix: %d, X: %d, Y: %d, Z: %d",m,x,y,z))
end
gui.register(whereami)
it would be really really really awesome if someone could extend this to show the id of the map to your left, right, top, and bottom, and how many steps away those are...  );

Nice! I'm running it on Pearl and the map matrix works perfectly! On the other hand the coordinates only change when I save, so I don't think they're reading the right thing?
15
I guess that gen4 has a rudimentary ASLR, like gen3 does.

Has anyone bothered doing major reversing of gen4 yet, both static analysis and debugging (setting in-memory breakpoints etc)?

While looking into it I found this list on Project Pokemon, and the 50 page gamefaq thread it links to. The Project Pokemon forum might have more threads where they researched this for their list, but I don't see them at a glance and their search is a bit annoying. It looks like they've moved on to 5th and 6th gen now.

There are tools available to change a lot on the ROM hacking side, but I'm not finding more than that about how the engine works.
Pages: [1] 2