What happened on the Day of Pentecost?
On the Day of Pentecost the Holy Ghost fell on the believers gathered in the upper room, they spoke with other tongues, and Peter preached that this outpouring was the fulfillment of the prophet Joel's words about God pouring out His Spirit on all flesh in the last days.
On the Day of Pentecost, the disciples, along with the women and Mary the mother of Jesus, were gathered together in an upper room in one accord in prayer. Suddenly a sound came from heaven like a rushing mighty wind, filling the house, and cloven tongues like fire appeared and sat upon each of them. They were all filled with the Holy Ghost and began to speak with other tongues, actual languages given by the Spirit, not their own native speech, as the Spirit gave them utterance.
The crowd that witnessed this was bewildered. Some were amazed, others mocked them as being drunk with new wine. In that charged moment Peter stood and preached, declaring that what was happening was exactly what the prophet Joel had spoken of: "And it shall come to pass in the last days, saith God, I will pour out of my Spirit upon all flesh: and your sons and your daughters shall prophesy, and your young men shall see visions, and your old men shall dream dreams: And on my servants and on my handmaidens I will pour out in those days of my Spirit; and they shall prophesy." Peter was not pointing to some far off future event; he was saying, "This is happening right now, today." Notably, God included women, calling them handmaidens who would receive the Spirit and prophesy, a radical and inclusive declaration for that time.
Pentecost also marked the beginning of a mystery, the moment the Holy Spirit first entered the bodies of believers, causing them to be seen as evidence that they had been with God before the world began. It was the start of the restoration of the members of the fellowship and the moment God began pouring His own life into human vessels. When the crowd asked, "men and brethren, what shall we do," the answer was to repent and receive this same gift, and each person who received the Holy Spirit represented, in a sense, the multiplication of Christ's presence, the single grain becoming many.
Peter's sermon that day also became the first message preached after the outpouring, and it later opened the door further when the Holy Ghost fell upon Gentiles as well while Peter was speaking, showing that the promise of the Spirit poured out on all flesh included Jew and Gentile alike, though some among the apostles and brethren initially contended with Peter over this. Pentecost, then, was the birth of the Church, the beginning of a season that continues today, marked by sons and daughters prophesying, young men seeing visions, and old men dreaming dreams, as the ongoing experience of God's people.
The scriptures
Then returned they unto Jerusalem from the mount called Olivet, which is from Jerusalem a sabbath day’s journey. And when they were come in, they went up into an upper room, where abode both Peter, and James, and John, and Andrew, Philip, and Thomas, Bartholomew, and Matthew, James the son of Alphaeus, and Simon Zelotes, and Judas the brother of James. These all continued with one accord in prayer and supplication, with the women, and Mary the mother of Jesus, and with his brethren.
And when the day of Pentecost was fully come, they were all with one accord in one place. And suddenly there came a sound from heaven as of a rushing mighty wind, and it filled all the house where they were sitting. And there appeared unto them cloven tongues like as of fire, and it sat upon each of them. And they were all filled with the Holy Ghost, and began to speak with other tongues, as the Spirit gave them utterance.
But this is that which was spoken by the prophet Joel; And it shall come to pass in the last days, saith God, I will pour out of my Spirit upon all flesh: and your sons and your daughters shall prophesy, and your young men shall see visions, and your old men shall dream dreams: And on my servants and on my handmaidens I will pour out in those days of my Spirit; and they shall prophesy: And I will shew wonders in heaven above, and signs in the earth beneath; blood, and fire, and vapour of smoke: The sun shall be turned into darkness, and the moon into blood, before that great and notable day of the Lord come: And it shall come to pass, that whosoever shall call on the name of the Lord shall be saved.
While Peter yet spake these words, the Holy Ghost fell on all them which heard the word. And they of the circumcision which believed were astonished, as many as came with Peter, because that on the Gentiles also was poured out the gift of the Holy Ghost.
And the apostles and brethren that were in Judaea heard that the Gentiles had also received the word of God. And when Peter was come up to Jerusalem, they that were of the circumcision contended with him, Saying, Thou wentest in to men uncircumcised, and didst eat with them.
From the books
This answer is drawn from these passages in the Mary Banks Faith Library.