Inclusion Criteria:
1. Patients must provide written informed consent prior to performance of study-specific procedures or assessments.
2. ARID1A Subgroup (N = 39):
1. Histologically confirmed locally advanced or metastatic solid tumor malignancy with progression on at least one prior systemic therapy, including one of the following tumor types:
* Renal cell carcinoma with predominant clear cell histology (Cohort A)
* Urothelial carcinoma (Cohort B)
* All pancreatic cancers (Cohort C)
* Other solid tumors excluding clear cell ovarian cancer and endometrial cancer (Cohort D)
* Endometrial and ovarian cancer (Cohort E)
2. Formalin-fixed paraffin embedded tumor tissue evaluable for BAF250a expression by ARID1A immunohistochemistry. Primary or metastatic tumor tissue is permissible. Patients without evaluable archival tissue may undergo optional tumor biopsy during Screening if other eligibility criteria have been met
3. Measurable disease by RECIST 1.1
3. ATM Loss Subgroup (N = 20):
1. Histologically confirmed locally advanced or metastatic solid tumor malignancy with progression on at least one prior systemic therapy, including one of the following tumor types:
* Metastatic castration resistant prostate cancer (N = 10).
* Patients may have evaluable or measurable disease by RECIST 1.1 criteria.
* Prior treatment with at least one androgen signaling inhibitor (e.g. abiraterone, enzalutamide, apalutamide, darolutamide).
* Patients will be required to maintain castrate levels of testosterone during study treatment with use of Luteinizing hormone-releasing hormone (LHRH) analog (except for patients with history of bilateral orchiectomy).
* Progression by PCWG3 criteria at study entry
* All other solid tumor malignancies (N = 10). Patients are required to have measurable soft tissue disease by RECIST 1.1 criteria.
2. Archival tumor tissue evaluable for ATM expression by immunohistochemistry (IHC)
3. Evidence of ATM loss by either pathogenic ATM mutation in Chemiluminescent immunoassay (CLIA)-approved assay and/or loss of ATM expression by IHC (Ventana Ab). An interim analysis will be performed after 10 patients are enrolled. If less than 50% of tumors have absence of ATM expression by IHC, subsequent enrollment of the remaining 10 patients will be required to have evidence of both ATM mutation and loss of ATM expression (\< 5% of tumor cells expressing ATM) using CLIA-certified IHC test (Ventana).
4. Endometrial Cancer Cohort (N = 30):
1. Histologically confirmed endometrial cancer
o A minimum of 15 patients must have the presence of pathogenic ARID1A alteration on CLIA-approved next-generation sequencing panel without evidence of microsatellite instability defined by next-generation sequencing and/or presence of intact mismatch repair proteins by immunohistochemistry.
2. Measurable disease by RECIST 1.1.
3. Availability of archival tumor tissue for retrospective testing of BAF250a expression by IHC.
4. Has received at least one prior line of systemic therapy for the treatment of locally advanced or metastatic disease, including progression on at least one prior line of therapy containing an immune checkpoint inhibitor that was administered for a minimum duration of 6 weeks
* Must not have experienced a toxicity that led to permanent discontinuation of prior immune checkpoint inhibitor.
* All adverse events (AE) while receiving prior immune checkpoint inhibitor must have completely resolved or resolved to baseline prior to screening for this study.
* Must not have experienced a \>= Grade 3 immune related AE or an immune related neurologic or ocular AE of any grade while receiving prior immune checkpoint inhibitor. NOTE: Patients with endocrine AE of \<=Grade 2 are permitted to enroll if they are stably maintained on appropriate replacement therapy and are asymptomatic.
* Must not have required the use of additional immunosuppression other than corticosteroids for the management of an AE, not have experienced recurrence of an AE if re-challenged, and not currently require maintenance doses of \> 10 mg prednisone or equivalent per day.
5. Body weight \>30 kg
6. No active or prior documented autoimmune or inflammatory disorders (including inflammatory bowel disease \[e.g., colitis or Crohn's disease\], diverticulitis \[with the exception of diverticulosis\], systemic lupus erythematosus, Sarcoidosis syndrome, or Wegener syndrome \[granulomatosis with polyangiitis, Graves' disease, rheumatoid arthritis, hypophysitis, uveitis, etc\]). The following are exceptions to this criterion:
* Patients with vitiligo or alopecia.
* Patients with hypothyroidism (e.g., following Hashimoto syndrome) stable on hormone replacement.
* Any chronic skin condition that does not require systemic therapy.
* Patients without active disease in the last 5 years may be included but only after consultation with the Principal Investigator.
* Patients with celiac disease controlled by diet alone.
5. Evidence of clinical or radiographic progression prior to study entry (except metastatic castrate-resistant prostate cancer (mCRPC) cohort which requires progression by PCWG3 criteria).
6. Age \>= 18 years at time of signing informed consent form.
7. Resolution of all prior treatment-related toxicities to grade 1 severity or lower (except alopecia).
8. Patients must be at least 3 weeks or 5 half-lives (whichever is shorter) from last standard or experimental non-cytotoxic therapy prior to first dose of protocol therapy. Patients must be \> 21 days from last dose of cytotoxic chemotherapy prior to C1D1. The minimum wash-out period for immunotherapy is 42 days prior to C1D1 with the following exception for the Endometrial cohort: Note: Washout from prior immunotherapy is \>=21 days to C1D1 for the Endometrial cohort.
9. Radiation therapy must be completed \> 7 days prior to course 1 day 1 (C1D1) or \> 28 days prior to C1D1 for patients receiving radiation to more than 30% of bone marrow.
10. Adequate organ function as defined by:
* Hemoglobin (Hgb) \>= 9.0 g/dL in the absence of transfusion within 14 days prior to screening laboratory assessment.
* Platelets (Plt) count \> 100,000 x 10\^9/L.
* Absolute neutrophil count \> 1.5 x 10\^9/L.
* Estimated glomerular filtration rate (GFR) \>= 45 ml/min based on Cockcroft-Gault equation or 24 hour urine collection.
* Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) \< 2.5 x upper limit of normal (ULN) (\< 5x ULN in patients with known liver metastases).
* Total bilirubin \< 1.5 x ULN (direct bilirubin \< 1.5 x ULN in patients with known Gilbert's disease or UGT1A1 homozygote).
11. Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1.
12. The effects of ceralasertib and olaparib on the developing human fetus are unknown. For this reason and because ATR and PARP inhibitors as well as other therapeutic drugs used in this trial are known to be teratogenic, women of child-bearing potential and men must agree to use 2 highly effective forms of contraception prior to study entry and for the duration of study participation. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately.
1. Male patients who are sexually active must be willing to use barrier contraception for the duration of the study and for 1 week after the last study drug administration, with all sexual partners. Male patients must use a condom during treatment and for 6 months after the last dose of study drug(s) when having sexual intercourse with a pregnant woman or with a woman of childbearing potential. Female partners of male patients should also use a highly effective form of contraception for 6 months after the last dose of study drug(s) if they are of childbearing potential. True abstinence for either sex is an acceptable form of contraception and must be documented as such.
2. Women of childbearing potential must have a negative serum or urine pregnancy test within 7 days prior to C1D1 treatment. Evidence of postmenopausal status or non-child bearing status must be documented. Postmenopausal is defined as:
* Aged more than 50 years and amenorrheic for at least 12 months following cessation of all exogenous hormonal treatments.
* Documentation of irreversible surgical sterilization by hysterectomy, bilateral oophorectomy or bilateral salpingectomy but not tubal ligation, radiation-induced oophorectomy with last menses \> 1 year ago, chemotherapy-induced menopause with \> 1 year interval since last menses
* Amenorrhoeic for 12 months and serum follicle-stimulating hormone (FSH), luteinizing hormone (LH) and plasma oestradiol levels in the postmenopausal range for the institution for women under 50.
13. Ability to understand a written informed consent document, and able to comply with the protocol for the duration of the study including undergoing treatment and scheduled visits and examinations.
Exclusion Criteria:
1. History of secondary malignancy requiring treatment within 1 year prior to screening, with the exception of carcinoma in situ of the cervix, non-melanoma skin carcinoma, low/intermediate risk localized prostate cancer (=\< Gleason 7, =\< T2N0M0, and prostate-specific antigen (PSA) =\< 20 ng/mL at diagnosis) (not applicable for prostate cancer cohort), ductal carcinoma in situ, Stage I uterine cancer, and non-muscle invasive urothelial carcinoma
2. Patients receiving, or having received within 14 days of C1D1, corticosteroids at a dose \> 10 mg/day of prednisone (or equivalent).
3. Patients with myelodysplastic syndrome or features suggestive of myelodysplastic syndrome.
4. Prior treatment with ATR inhibitor
5. Major surgical procedures \< 28 days prior to C1D1. Patients must have recovered to grade =\< 1 for any adverse events related to the surgical procedure.
6. Untreated central nervous system (CNS) metastases. Patients with previously treated central nervous system (CNS) metastases are eligible if:
* No requirement for corticosteroids at study entry
* Radiographically and clinically stable for at least 4 weeks prior to study entry
* No evidence of intra-tumoral hemorrhage
* No evidence of current or prior leptomeningeal disease.
7. Clinically significant gastrointestinal abnormalities that may increase the risk of decreased absorption of medications, including:
* Inability to swallow oral medications
* Active peptic ulcer disease
* Known intra-luminal metastatic lesions
* History of abdominal fistula or bowel perforation
* History of bowel obstruction within 6 months prior to study entry
* Known malabsorption syndrome
* Significant resection of the small bowel.
8. Fridericia's QT correction formula (QTcF) \> 470 ms (females) or \> 450 ms (males) on screening electrocardiography (ECG), or immediate family history of congenital long QT syndrome or sudden cardiac death at age less than 40.
9. History of any one or more of the following cardiovascular conditions within the past 6 months:
* Myocardial infarction
* Unstable angina
* Transient ischemic attack or cerebrovascular accident
* Uncontrolled arrhythmia. Rate controlled atrial fibrillation/flutter is not an exclusion for the study.
* Class III or IV congestive heart failure or documented left ventricle (LV) ejection fraction of \< 50% (screening not required).
10. Uncontrolled hypertension as defined by systolic blood pressure \> 160 mm Hg and/or diastolic blood pressure \> 100 mm Hg. Adjustment of anti-hypertensive regimen and re-screening is permitted.
11. Relative hypotension with resting blood pressure of less than 90 mm Hg systolic and less than 60 mm Hg diastolic or symptomatic orthostatic hypotension.
12. Any serious and/or unstable pre-existing medical, psychiatric, or other condition that could interfere with patient's safety or adherence to study procedures including uncontrolled infection requiring parenteral antibiotics.
13. Concomitant use of strong cytochrome P450, family 3, subfamily A (CYP3A4) inhibitors, strong CYP3A4 inducers, CYP3A4 substrates with narrow therapeutic index, or CYP2B6 substrates with narrow therapeutic index within 21 days or 5 half-lives, whichever is shorter, prior to C1D1 of study treatment
* The use of herbal supplements or 'folk remedies' (and medications and foods that significantly modulate CYP3A activity) should be discouraged. If deemed necessary, such products may be administered with caution and the reason for use documented in the case report form (CRF).
14. A known hypersensitivity to olaparib, ceralasertib, durvalumab (as applicable to the study drugs the patient is receiving), or any excipient of the product or any contraindication to the combination anti-cancer agent as per local prescribing information.
15. A known chronic active hepatitis B or C (defined by positive viral load; screening not required).
16. Immunocompromised patients, including those serologically positive for human immunodeficiency virus (HIV), those receiving chronic immunosuppression, or those with prior allogeneic or cord blood transplantation.
17. History of myelodysplastic syndrome (MDS) or acute myeloid leukemia (AML).
18. Presence of prolonged severe cytopenia (≥ Grade 3 for ≥ 2 weeks) prior to enrolment.
Active drug
